He died of renal failure in a Los Angeles hospital with his family by his side, his spokesman, Harry Flynn, told the Associated Press.
Mr Borgnine, who was 95 years old, continued acting until recently with a role in the hospital drama ER in 2009.
He won an Oscar in 1955 for his role in the film Marty.
His family released a statement saying Borgnine "had been in excellent health until a recent illness".
Borgnine was also known for his roles in western The Wild Bunch and disaster movie The Poseidon Adventure.
Borgnine was considered one of Hollywood's leading character actors
His TV series McHale's Navy was also a major success in the United States. He is also remembered for his role as the jovial Dominic Santini in the 1980s series Airwolf.
His manager Lynda Bensky called it "a very sad day", adding: "The industry has lost someone great, the calibre of which we will never see again.
"A true icon. But more importantly, the world has lost a sage and loving man who taught us all how to 'grow young'".
Fellow stars have been paying tribute on Twitter, including his friend, the Oscar-winning actress Marlee Matlin.
"So sad to read about passing of Ernest Borgnine," she wrote. "We spoke recently at Paramount's 100th anniversary photo. A true legend & a gentleman. RIP."
Cheers star Kirstie Alley tweeted: "Ernest Borgnine... You will be missed... I'm blessed to have worked with you...RIP sweet, talented friend."
Joe Mantegna, who starred in The Godfather Part III and TV series Criminal Minds said: "I cannot express my sorrow at the loss of Ernest Borgnine. He was a true Hollywood icon and a dear friend. My prayers go out to the family."
British actor Mark Gatiss said: "Farewell to the mighty Ernest Borgnine. A lovely career & working till the end! Can we show 'Marty' on TV, please?"
Others to pay tribute included James Corden, Red Hot Chili Peppers bassist Flea, William Shatner, Ugly Betty star Michael Urie and former Happy Days actor Scott Baio, who called him, "a great Italian man, actor, Navy veteran, & father".
Writer and broadcaster Andrew Collins said: "I met Ernest Borgnine about ten years ago. He lit up the room. What a privilege. And he was in The Poseidon Adventure. "A lotta guts." RIP"
The son of Italian immigrants, he was born Ermes Effron Borgnino on 24 January 1917 in Hamden, Connecticut. His parents later changed the family name to Borgnine.
He initially never considered a career in acting, serving in the US Navy after leaving high school in 1935.
It was only on the prompting of his mother than he enrolled in a drama school after the end of World War II.
After making his acting breakthrough in a Tennessee Williams' stageplay, The Glass Menagerie, Borgnine gained his first major film role in From Here to Eternity in 1953, playing a sadistic sergeant who beats up Frank Sinatra's character, Private Angelo Maggio.
Borgnine was presented with his Oscar by Grace Kelly in 1955
His trademark stocky build, gruff voice and leering grin led him to be frequently cast as the villain.
But he escaped the stereotype in 1955, landing an Oscar-winning lead role as a romantic New York butcher in the low-budget film Marty.
He went on to play character-acting roles in more than 60 films, including The Vikings (1958), The Flight of the Phoenix (1965), The Dirty Dozen (1967), Sam Peckinpah's The Wild Bunch (1969) and "The Poseidon Adventure" (1972).
Borgnine's work in later life included playing the voice of Mermaid Man on children's TV series SpongeBob SquarePants, as well as that of Carface on the animated film All Dogs Go to Heaven 2.
His personal life was turbulent at times, and included four failed marriages. One, to singer Ethel Merman, lasted barely a month.
"The Oscar made me a star, and I'm grateful," Borgnine said in 1966. "But I feel had I not won the Oscar, I wouldn't have gotten into the messes I did in my personal life."
However, his fifth marriage, to Norwegian-born businesswoman Tova Traesnaes in 1972, endured. He is survived by three children from two earlier marriages.