Now, after the Chinese military building had kept a low profile for years, its brief debut in the city's beloved "Symphony of Lights" felt like nothing less than a show of force 17 years after the British handed the territory back to Chinese control.
"It's a logo of red Chinese colonization," said Billy Chiu Hin-chung, one of four people arrested last year after storming the army building while waving Hong Kong's colonial British-era flag.
"If Hong Kong people don't obey the Communist Party," Chiu predicted, "the army will come and fight us."
From the sweltering streets of this legendary port city of 7.2 million people to its air-conditioned office towers, Hong Kongers are indeed picking sides in a looming battle over their city.
People here have long prided themselves as providing a stable, sophisticated alternative to Communist China that despite its small population enjoys the world's 36th-biggest economy and runs the globe's sixth-richest stock exchange.
But now, Hong Kongers say the soul of their society is coming under attack as they see the flood of cross-border Chinese shoppers (dubbed "locusts" for their voracious buying habits and supposed bad manners) and grow wary of the Communist Party's rising sway with top officials.
One significant fear is that Beijing is breaking promises to let voters elect their leaders for the first time starting in 2017. Chief Executive Leung Chun-ying, who was hand-picked by a committee of mostly pro-Beijing elites, recently asked China's legislature for constitutional changes to allow the territory to pick its own leader. However, his report said "mainstream opinion" wanted the committee to again pick candidates, setting the stage for a confrontation with democracy groups.
Already, the pro-Beijing influence is threatening a disciplined civil service corps traditionally untainted by political corruption, says Anson Chan, a democracy activist who was Hong Kong's chief secretary and No. 2 official from 1993 to 2001.
"If the government gives the community the impression that it doesn't listen," she says, "then the community feels that the only way of making this government listen is to take to the streets."
In the eyes of Chan and others, Beijing's influence has also hit the city's media industry. Most newspapers no longer run stories critical of the Chinese government, and in February an outspoken former editor at the Ming Pao newspaper was attacked by cleaver-wielding men.
Hong Kong's journalists' association called the past 12 months "the darkest for press freedom for several decades," citing the attack and advertising boycotts. Last year, the French press watchdog group Reporters Without Borders ranked Hong Kong 61st in press freedom, down from No. 18 in 2002.
"For someone who is used to an open society, that is something really alarming and concerning," says Shirley Yam, the journalists' association's vice chairwoman. "Hong Kong is a major financial center, and the reason that Hong Kong has been able to become a major financial center is freedom of information and press freedom."
The most troubling blow came last month with the white paper, which argued that Hong Kong's high degree of autonomy, famously dubbed "one country, two systems," was entirely at Beijing's discretion. It added that "loving the country is the basic political requirement for Hong Kong's administrators," including its judges.
Even Hong Kong's lawyers, a reserved group who dress for court in wigs and black robes, hit the streets by the hundreds to protest the white paper.