Just months ago, Geert Wilders was an anathema to most Dutch political parties.

A disruptive and divisive force on the far right for two decades, Mr. Wilders has said he wants to end immigration from Muslim countries, tax head scarves and ban the Quran. He has called Moroccan immigrants “scum.” His Party for Freedom has supported leaving the European Union.

But then Mr. Wilders won national elections convincingly in November. Nearly a quarter of Dutch voters chose his party, which won 37 of 150 seats in the House of Representatives, a huge margin by the standards of a fractious party system that rests on consensus and coalition building.

Since then, Mr. Wilders has become an unavoidable political force. “He is the biggest,” said Janka Stoker, a professor of leadership and organizational change at the University of Groningen, of Mr. Wilders. “They simply can’t ignore him.”

That quandary has made the Netherlands a test case for Europe as it grapples with the question of what to do with far-right forces that have advanced so far into the mainstream that they can hardly be considered on the fringe anymore.

Italy already has a hard-right leader, and the Swedish government depends on a party with neo-Nazi roots. The far right now represents significant parts of the opposition in France and Germany, forcing the question of how much longer it can be shunned.

In the Netherlands, some mainstream parties have answered by holding their noses and marching forward into the negotiating room to find a way to work with Mr. Wilders.

Coalition talks to form a new government, which have a history of taking weeks or months, broke down in February, not over anything specific Mr. Wilders said or did to further offend the political establishment, but over budget numbers.

It was a tellingly mundane obstacle that betrayed the deepening normalization of Mr. Wilders and his political acceptance by the other parties.

“His normalization has gone very fast,” said Cas Mudde, a Dutch political scientist at the School of Public and International Affairs at the University of Georgia.

“Most mainstream media and politicians have treated the coalition negotiations with Wilders as normal,” Mr. Mudde said, “which seems also the view of a majority of Dutch people.”

The parties on the left have said categorically that they still reject Mr. Wilders. But the question of how to govern with him is not for them; it is for parties across the rest of the political spectrum.

Mr. Wilders has been negotiating with the People’s Party for Freedom and Democracy, a center-right party that governed for the past 13 years; the Farmer Citizen Movement, a populist pro-farmer party; and New Social Contract, a new centrist party. Together, these four parties have 88 seats in the House of Representatives, a comfortable majority.

But the discomfort of Mr. Wilders’s negotiating partners is evident, even if they do not express it publicly.

The concerns swirling around Mr. Wilders remain such that early on in the talks, the four parties around the table took the unusual step of signing a document committing them to uphold the Dutch Constitution, something that had long been taken for granted.

The pledge, as well as the need to cobble together support from multiple parties, is expected to limit Mr. Wilders’s ability to drastically change any pillars of the Dutch government or to push through unconstitutional laws.

Signing the document also helped Mr. Wilders gain some political distance from his most extreme positions as he sought to find common ground with the other parties, lending the impression that he was moderating his views.

But Mr. Wilders’s party is built entirely around him and has a unique structure that gives him sole authority. Its platform still includes numerous unconstitutional proposals, including a ban on mosques and Islamic schools.

Mr. Wilders has said he has not changed his opinions, and he has refused to take back comments that have landed him in hot water. Those include his question to supporters in 2014 about whether they wanted more or fewer Moroccans in the Netherlands, which resulted in chants of “Fewer! Fewer!” from the crowd.

“Our vision and criticism on Islam hasn’t changed,” Mr. Wilders told lawmakers last month.

Given his refusal to disavow his most extreme statements, and the fact that he and his party are one and the same, the country and political establishment are left with the lingering issue of which Mr. Wilders they are dealing with.

“The question is: How believable is someone who hasn’t taken the Constitution seriously for about 20 years?” said Léonie de Jonge, an assistant professor in European politics and society at the University of Groningen. “If you look at his Twitter feed, then he is not less radical.”

At the same time, Mr. Wilders is a known quantity, one of the many contradictions that surround him. He is one of politics’ most familiar faces in the Netherlands and is known for blunt language. He has served in the House of Representatives since 1998, making him the longest-sitting member.

For most of that time, he has been a fixture of the opposition. But he managed his greatest success in the last election by tying his longstanding antipathy toward immigration to other issues Dutch citizens cared most about, like a shortage of affordable housing.

To try to find a way out of the impasse around forming a government — and to avoid everyone’s least favorite option, new elections — politicians have been talking about unorthodox arrangements with little or no precedent.

Some have floated the idea of forming a minority coalition, or a cabinet that could include ministers from other, smaller parties. It could also include political outsiders, potentially respected former politicians who would serve to create a greater buffer between the cabinet and the Parliament. But political analysts as well as politicians themselves are unclear on what that would mean in practice.

Yet even those options might only reduce Mr. Wilders’s role. They could prevent him from becoming prime minister, but in almost any conceivable circumstance, his party would have to be a part of the government. There is almost no getting around it.

During the round of coalition talks that collapsed last month, Mr. Wilders used his opportunity as the center of political attention to put his best face on and present himself as professional and constructive.

“I behaved myself as the leader of the biggest party,” Mr. Wilders said during a recent debate in the House of Representatives.

Even traditionally hostile parts of the Dutch news media have described him as reliable and professional, including a left-wing newspaper, De Volkskrant, that has long been highly critical of him.

Yet Mr. Wilders has continued to be combative on social media, a favorite platform for his most vitriolic outbursts. His behavior continues to raise questions about his ability to serve as a uniting force, a side of Mr. Wilders that has been neither seen nor tested during his time in the opposition, according to Dr. Stoker.

For now, the four negotiating parties will have to resume talking and come to a consensus about which kind of coalition structure they can support before trying to hash out a governing agreement.

Though the chances of a traditional coalition with Mr. Wilders at the helm may dwindle in a new round of talks, that option remains.

Mr. Wilders himself says he wants to lead the country. When asked in a recent parliamentary debate if he was still willing to become prime minister, his answer was clear: “I can’t wait.”