The new mayor gave his speech just moments after his public swearing-in ceremony.