Rudi Schmitt was born in Frankfurt am Main on January 8, 1928.

After completing his studies, he worked as a teacher in Frankfurt from 1946 until his election as an SPD member of the Hessian state parliament in 1954. From 1960 to 1968, Schmitt worked as a city councillor in Wiesbaden as head of the schools and sports department and, from 1966, also as head of the cultural department. He resigned from these posts after being elected Lord Mayor of the state capital on February 1, 1968.

After leaving the town hall, he was elected to the German Bundestag for two legislative periods until 1987 via a direct mandate.

Wiesbaden bid farewell to its former mayor on March 14, 1980 with an entry in the city's Golden Book.