Mattis will be the next secretary of defense, while Kelly will serve as the secretary of the Department of Homeland Security.
Their confirmations mark Trump's first Cabinet nominees to receive Senate confirmation.
Mattis' nomination and confirmation was one of the least controversial of Trump's picks.
But before he could be confirmed, he needed a waiver to be able to serve in the role thanks to a rule that requires the secretary of defense to have been retired from the military for at least seven years. Mattis retired from service in 2013.
Kelly too was a noncontroversial Trump nominee, seen as more moderate than other contenders Trump was eyeing for the role.
During Kelly's confirmation hearing, he broke with Trump on a number of key issues, saying he was against torture and expressed doubt for the need to build a wall on the United States' border with Mexico.
More of Trump's Cabinet nominees will receive a vote in the coming days, including the nominee to lead the CIA, Calif. Rep. Mike Pompeo, who will receive a vote on Monday.
Democrats in the Senate sought a delay on Pompeo's confirmation, asking for more time to vet him as the CIA nominee.