If I pay a collection agency account, will it still count against my credit score?

Before very long, it should not. FICO (formerly Fair Isaac & Company), the company that invented the FICO credit scoring model, issued a Press Release on August 9, 2014. It announced that paid collection agency accounts will be "bypassed" in determining credit scores, under it's new FICO credit scoring algorithm or formula.

This will likely take a while to be fully implemented, but it should be a big help for some consumers trying to recover their good credit score after having financial difficulties. FICO also announced that medical collections won't count as much against consumer's scores, as other types of items.