Mr Albanese’s Labor Party defeated Scott Morrison’s conservative government in an election on Saturday.
It remains unclear whether Mr Albanese will form a majority or govern with the support of crossbenchers.
The prime minister left for Tokyo on Monday to meet the leaders of the so-called Quad nations – the US, India and Japan.
Earlier in the day, he was sworn in with four key cabinet members, including new Foreign Minister Penny Wong, who is travelling with him to Japan.
Richard Marles is the new deputy prime minister and employment minister, Jim Chalmers is treasurer, and Katy Gallagher is attorney-general and finance minister.
It is Australia’s first Labor government in almost a decade. The party has won 72 lower house seats but counting continues to determine whether they can get the 76 needed to form a majority.
But the primary vote for both major parties fell – almost a third of Australians put the Greens, independents and other minor parties as their first preference.
The Quad group is seen as largely aiming to counter growing Chinese influence in the Indo-Pacific region.
It will meet on Tuesday following recent diplomatic tensions in the Pacific, after the Solomon Islands last month signed a security pact with China.
The US and Australia hold fears the deal could allow China to build a naval base there.
In a statement ahead of the meeting, Mr Albanese said: “The Quad Leaders’ Summit brings together four leaders of great liberal democracies – Australia, Japan, India, and the United States of America – in support of a free, open and resilient Indo-Pacific.”
Ms Wong – Australia’s first overseas-born foreign minister – signalled they would bring “new energy and much more to the table” on climate action, after “a lost decade”.
Climate change played a huge role in the election result, with a surge in support for candidates wanting urgent action.
Mr Morrison’s government had committed to a 2030 emissions reduction target of 26%-28% – about half that of the UK and US. Mr Albanese’s government has a target of 43%.
The Greens are expected to pick up four lower seats, adding to seven climate-focused independents. They could put pressure on Labor to take even stronger action, especially if it fails to reach a majority.
Mr Morrison’s unpopularity and his party’s stance on climate have been blamed by some Liberal MPs for wiping out their vote.
Losses included senior party figures, including deputy leader Josh Frydenberg, in traditional Liberal strongholds.
Mr Morrison stepped down as party leader on Saturday and former defence minister Peter Dutton is the favourite to succeed him.
Mr Dutton – from the party’s right – has been a controversial figure at times. Some question whether he could rebuild Liberal support in more progressive, metropolitan areas.
It’s been a busy few days for Mr Albanese. Since Saturday he has won an election, been sworn in as PM, and is now on his way to the Quad.
It’s fitting that Australia’s new leader would hit the ground running. There’s a lot to get to domestically and globally.
Mr Albanese has said the Quad alliance is an absolute priority for Australia. This meeting comes at a crucial and tense time in the region with a growing Chinese influence that’s making Australia and its allies nervous.
The recent China defence deal with the Solomon Islands is seen as a threat to Australia’s status as a strategic partner to Pacific nations. But there’s a fine balance Mr Albanese needs to strike.
He needs to address China’s expansion in the Indo-Pacific while also repairing a relationship with Beijing that’s fractured significantly since the beginning of the pandemic when Australia called for an investigation into the origins of the Covid virus.
China is now a very angry strategic trade partner. And the other Quad members will need to be reassured that Mr Albanese has a plan to manage the complicated relationship.
The prime minister says this trip is a chance for Australia to send a message globally that it is changing its approach on crucial policies such as climate change.
This is now being called the climate election. Mr Albanese wants to tell his allies and the world that under his leadership they’ll be dealing with a new and more globally-engaged Australia.