Unless you've been invited by the administration of the serving Prime Minister, the closest you'll get to Downing Street are the steel railings at the Whitehall end of the street. The serving Chancellor of the Exchequer occupies the building next door at number 11.