Johnson was born in Canton, South Dakota, and raised on a farm. He graduated from the University of South Dakota in 1982 with a degree in political science. After working for several years as a legislative aide, he was elected to the South Dakota House of Representatives in 1986. He served in the House for six years, including two years as speaker.
In 1996, Johnson was elected to the United States Senate, defeating incumbent Republican Larry Pressler. He was reelected to a second term in 2002 and a third term in 2008.
Johnson was known for his moderate political views and his willingness to work with both Republicans and Democrats. He was also a strong supporter of agriculture and rural development.
In 2014, Johnson announced that he would not seek a fourth term in the Senate. He cited health problems, including a stroke he had suffered in 2006, as the reason for his decision.
Johnson left the Senate in 2015. He is now retired and lives in Vermillion, South Dakota.