The Bell pub opposite the Brookwood Triangle garden in Webber Street is to be demolished to make way for a development of eight new flats.
Southwark Council's planning committee approved plans for a five-storey building to replace the two-storey pub on Tuesday evening.
The former Charrington's pub is flanked by The Priory, birthplace and childhood home of Picture Post photographer Bert Hardy, and Friars Primary School.
The committee took an hour to come to its decision following a discussion on the historic importance of the pub and a lack of amenity space for future residents in the plans before the committee.
The present early 19th-century building is sometimes claimed as the successor to The Windmill in St George's Fields mentioned by William Shakespeare in Henry IV part 2.
Applicant David Kaye of Ebury Holdings said that the pub had not been operating for over a year since when it had been squatted by "some undesirable persons" including some wanted by the police.
Mr Kaye claimed that his proposal would improve the area and be in keeping with surrounding buildings.
Asked about the amenity space not being policy compliant, he said that the site was small and reducing the number of flats to allow for more outdoor space would mean that the development ceased be financially viable.
Michael Tsoukaris, Southwark Council's conservation officer, described The Bell as "a "fine building" having "charm and distinction" and being "of communal interest".
However, the property did not lie within a conservation area and had not been listed by English Heritage.
Although it appears on Southwark Council's own list of buildings of historic interest, this remains a draft document since it has never been put to public consultation. Legal advice given at the meeting was that the local listing could not be deemed a 'material consideration'.
Objections to demolition were submitted by neighbours in Tadworth House who said: "The Bell public house is an attractive building which enhances the area – another block of flats is just another block of flats."
Other residents pointed out that The Bell was in keeping with other historic buildings on the same side of the street including The Priory, Hope Mission chapel and Blackfriars Foundry.
An objection was also received from CAMRA, the Campaign for Real Ale.
"I am not comfortable with it," commented Cllr Robin Crookshank Hilton who wondered if the pub could not have been converted into flats rather than demolished.
Cllr Althea Smith said that developers should think about community and the area.
The committee voted 4 to 2 to approve the application.
For the latest local news and events direct to your inbox every Monday, you need our weekly email newsletter SE1 Direct.