The combination of CIMB, RHB Capital and Malaysia Building Society (MBSB) would leapfrog Maybank, the country's largest bank by assets.
It would also mark a fresh wave of consolidation in the sector as banks in the region jostle to take advantage of closer financial integration among the 10 members of the Association of Southeast Asian Nations.
The banks said they had received regulatory approval to start negotiations with the aim of merging RHB and CIMB, as well as creating an enlarged Islamic banking franchise with MBSB.
The combined entity would have assets of Rm597 billion ($188 billion) as of the end of 2013, overtaking Maybank's Rm560 billion. It would also rank as Asean's fourth-largest bank after Singapore's top three of which DBS – Singapore's largest – has assets of S$419 billion (US$337 billion), according to Bloomberg data.
Robust domestic economies, relatively high savings rates and the emergence of companies with regional ambitions have turned Asean, with a combined population of about 650 million, into a lucrative banking market.
Malaysia, the bloc's third-largest economy after Indonesia and Thailand, is also by far the largest centre in Asia for Islamic finance, driven by its majority Muslim population and strong financial and political links with the Middle East.
Islamic banking assets in the country totalled Rm434 billion at the end of May, representing 21 per cent of total system-wide assets, compared with 16 per cent at the end of 2009, according to Moody's.
It estimates that the Rm3.25 billion of ringgit-denominated sukuk issued by Islamic banks in Malaysia in the first half of the year was the largest in any year so far.
"The opportunity to be a part of a mega Islamic bank is exciting for us and we want to take this forward," said Ahmad Zaini bin Othman, MBSB's chief executive.