The Frenchman had been expected to move into a new role within the company following a management shake-up at the rebranded Alpine squad, but instead he is departing completely.
Rossi will take charge of Alpine cars, its F1 team and other motorsport activities. He will report directly to Luca de Meo, Renault's CEO.
It is expected that Abiteboul's position as team principal of the F1 squad will be taken by Marcin Budkowski.
Speaking about his departure, Abiteboul said that he felt that he was leaving behind a strong base for the French car manufacturer to grow.
"I would like to thank the Groupe Renault for having trusted me for many years, particularly with the relaunch and reconstruction of the team since 2016," he said.
"The solid foundations of the racing team and the entities in France and England built over these years, the strategic evolution of the sport towards a more economically sustainable model, and more recently the Alpine project which provides a renewed sense of meaning and dynamism, all point to a very fine trajectory."
De Meo added: "I would like to warmly thank Cyril for his tireless involvement, which notably led the Renault F1 Team from the penultimate place in 2016 to the podiums last season.
"His remarkable work in F1 since 2007 allows us to look to the future, with a strong team and the new Alpine F1 Team identity to conquer the podiums this year."
Abiteboul is a Renault stalwart who originally joined the company after university. He began working for the F1 team from 2007 before moving up to become its executive director in 2010.
He joined Caterham in 2012 and became its team principal, before his departure in 2014.
He returned to Renault as managing director and played a key role in the French car manufacturer's return to F1 as a works team in 2016.