Charles B. Rangel died Monday at age 94, leaving behind a larger-than-life legacy in Harlem, his birthplace and longtime home, which he represented in Congress for more than four decades.

To veterans, friends and Harlem residents who gathered on Monday for a Memorial Day lunch at American Legion Post 398, a few blocks from his home, he was just Charlie: a onetime member of the Legion post, a political powerhouse who always made himself accessible to his constituents.

Nadine Pittman, a longtime American Legion Auxiliary member and a lifelong Harlem resident, described Mr. Rangel as “down-to-earth with the people.”

“He’d take the time and talk to you,” Ms. Pittman said. “I loved him as a person.”

Mr. Rangel retired as the ninth-longest continuously serving member of the House of Representatives in U.S. history. He was part of a quartet of venerable Harlem politicians known as the Gang of Four.

Mr. Rangel was born and raised in Harlem and attended DeWitt Clinton High School in the Bronx until he dropped out to join the Army in 1948. He fought in the Korean War and was awarded a Bronze Star for valor after leading his all-Black unit to safety.

Mr. Rangel was elected in 1966 to the State Assembly. In 1970, he was voted into Congress, unseating Adam Clayton Powell Jr., a longtime incumbent.

He remained in Congress for more than four decades. In 1974, he became the first Black member of the Ways and Means Committee and then, more than three decades later, its first Black chairman. He lost that powerful post four years later, in 2010, after he was censured for ethical lapses, including tax improprieties and failing to disclose personal income. Despite the scandal, he continued to win re-election and stayed in Congress until his retirement.

Throughout his lengthy career, Mr. Rangel never left behind the Harlem neighborhood where he had been raised. For years until his death, he lived in an apartment in Lenox Terrace, not far from where he had grown up.

“He would walk down the street and everyone would go, ‘Hi, Charlie,’” said G. Rod Roderick, a Vietnam veteran and a former commander of Post 398.

He came to know Mr. Rangel well after years of crossing his path at Memorial Day parades, Junior R.O.T.C. events and weekly jazz concerts in the Legion hall’s basement. Mr. Rangel liked jazz, Mr. Roderick recalled, but it was his wife, Alma Rangel, who really loved it.

William Frieson, a Navy veteran, said he first met Mr. Rangel in 1977. The two became dear friends, despite what Mr. Frieson described as “a difference of political opinion.”

“His political so-and-so is different than our friendship,” he said. “He was a hell of an individual.”

In recent years, Mr. Rangel stopped showing up at the post as frequently. But members still saw him, sometimes at Sylvia’s, the mainstay Harlem soul-food restaurant where he had been a regular, or outside Lenox Terrace.

“I like that about him,” said Diane Walters, an auxiliary member serving lunch upstairs at the Legion post. “He stayed in his community.”

The congressman was linked inextricably to Harlem, Mr. Roderick said.

“When you said ‘Harlem representative,’ the first thing that came to mind was Charlie.” Mr. Roderick said. “You never thought of anyone else.”

“We just considered him part of the family.”