Want a great credit card without a big annual fee? Here's the secret to paying a lot less.

A classic conundrum for people seeking rewards credit cards is that the most generous cards tend to come with hefty annual fees. Yet it turns out there is a solution, according to CreditCards.com: Just call.

A new poll from the site found that only 1 in 4 cardholders attempt to contact their credit card company to make a fee or credit limit request. But of those who did bother to call and ask, 82% said they had their annual fee waived or reduced. Plus, 87% were able to get a late fee waived, 69% lowered their interest rate and 89% received a higher credit limit — which is one smart way to boost your credit score.

Why aren't people asking? "People don't try this often because they don't think they will be successful," CreditCards.com senior analyst Matt Schulz said in a phone interview. "Our survey clearly shows if people take the time to ask, they have a really good chance of having their request granted."

One caveat: Easy relief from fees by making a simple phone call may not be possible in a few years, so Schulz advised consumers to act now. 

"Credit card debt is rising, interest rates are on the increase and we are seeing delinquency rates rising a bit as well," he said. "The credit card world is slowly changing, so now is the time to ask for those breaks, because banks aren't going to be this giving forever. We suspect two-to-three years down the road, depending upon the economy, these breaks may not be as easy to obtain."

CreditCards.com commissioned Princeton Survey Research Associates International to poll 952 credit card holders living in the continental United States about their success rate obtaining credit card cost breaks.

Which credit card rules are easy to change?

Younger millennials — those between the ages of 18 and 26, who are typically building credit history for the first time — were particularly successful at having their limits raised, with 80% being successful when they asked: "It's great younger millennials are asking for higher credit limits," Schulz said. "Their first card probably had a crummy interest rate and a low credit limit, so upgrading is smart." Schulz also noted that obtaining a higher limit can improve your utilization rate.

Younger baby boomers, between the ages of 53 and 62, were the most aggressive and successful at having annual fees waived. Among those who asked, 19% had their fee lowered and 78% had their fee completely waived. Boomers also had the highest success rates when they asked for other breaks, which makes sense: Longer credit histories mean more bargaining power.

But even some millennials are old enough to benefit from the perks of seniority, it seems: 93% of those ages 27 to 36 who asked for any of the perks or fee waivers from their credit card company were successful at least once — the highest overall success rate of any demographic, CreditCards.com pointed out.

How to pay less for your credit card

Definitely make a phone call and ask for a break. Just know when and how to approach each situation, Schulz advised. "When it comes to the late fee, you will probably be successful with a first request as long as you are polite," he said. "However, you probably aren't going to be successful by the third or fourth time making this request."

Also, understand that premium credit cards are a different animal than other cards. CreditCards.com found 51% were able to have their premium credit card annual fee waived when they asked and 31% had their fee reduced, but certain in-demand cards will require more sweet-talking. "I would not expect to call Chase and get the $450 annual fee on the Sapphire Reserve waived," Schulz said. "However, there are strategies you can use."

One example? Schulz said he had an annual fee waived on an airline credit card.

"I stopped using the card after I received the signup bonus," he recalled. "I liked the perks but didn't want to pay the $95 annual fee. When I called, the credit card company worked it out with me where if I spent $95 on the credit card, they would waive the fee. I would have spent the $95 anyway, so I ended up saving money."

Bottom line — it never hurts to ask. "Just like what my dad told me, what's the worst thing that can happen if you ask?" Schulz said. "They just say no."

Sign up for The Payoff — your weekly crash course on how to live your best financial life. Additionally, for all your burning money questions, check out Mic's credit, savings, career, investing and health care hubs for more information — that pays off.