The Japanese manufacturer introduced a new engine for this year ahead of its departure from the sport at the end of the season, before Red Bull takes over its development from 2022.
The new engine is smaller and more powerful, and both Red Bull and AlphaTauri have taken a step forward judging by their performance in pre-season testing and in the first race in Bahrain.
"I must say Honda did a fantastic job in Sakura, because this new power unit is much more powerful and better drivable as it was in the past," said Tost in Bahrain, where Pierre Gasly qualified fifth.
"I think that Honda is really very, very close to Mercedes. And I can only say thank you to the Japanese engineers, because they did a fantastic job."
Gasly struggled in the race after damaging his front wing and floor in an incident with McLaren's Daniel Ricciardo at the start. The French driver retired his car with four laps to go, but rookie teammate Yuki Tsunoda finished ninth on his grand prix debut.
Despite leaving Bahrain with just two points, Tost remains convinced that AlphaTauri has progressed in all areas, although he concedes the midfield group is extremely tight this year.
"We improved in every aspect," he said. "But we must not forget our competitors as well. If you look to McLaren, they have the Mercedes power unit, they are very strong.
"For me, currently, Red Bull, Mercedes are on the same level, maybe Red Bull is even faster. Then I see McLaren coming. And we see then what's going on with Ferrari. But we are close to Ferrari and to the rest.
"It's a question of hundredths of a second. We improved but the others improved as well. The question is now who did a better job? And this I can answer at the end of the season."
He added: "I need two, three races to get a better picture. But it's very tight and I think it will be the same game as last year."
Tost sees no reason why the positive trend can't continue in the second race of 2021 at Imola.
"I expect the same for Imola. We will then see what's going on in Portimao," Tost said. "But generally speaking the car is fast, hopefully also reliable, and we have a strong package together.
"The team is becoming more experienced now. We fortunately didn't have any fluctuation from the engineering side, which is very important, and also on the operational side we have a strong crew together now.
"And we have two real fast drivers. Pierre, with more experience, and Yuki as a newcomer, as a rookie, but I think that he is learning fast and this combination should help us to bring together a more competitive package as it was last year."
The team scored its first F1 win under the AlphaTauri name last year, when Gasly triumphed in the Italian Grand Prix.
Additional reporting by Erwin Jaeggi