Bibliometrics, Altmetrics and research quality

How do you measure research quality? This is a question the university world and its funding bodies have been asking for many years, but which I have only just developed an interest in.  

I attended a short course on Research Support for academics a few weeks ago and the issue of how universities, governments and researchers prove the quality of their research output came up a number of times and we spent a full day discussing it as well (and the wide variety of tools that purport to measure some aspect of it). 

I can certainly see the importance of research quality to universities, governments and funding bodies and the reasons why proving it in the ‘acceptable’ ways has become a key driver for researchers, but I do worry that by instituting these measures of quality, we are actually just creating a system that only values a particular type of research.

Current measures of research quality include the journal impact or rank, citations (per paper, per person, per year and more), collaborations and peer review quality, as well as how many people are talking about a paper, viewing it, and downloading it. And although the creators of the various scores and tools don’t claim that they are the ultimate resource for assessing quality, this often looks like it is overlooked in the mad rush to prove that researchers are providing value-for-research dollars.

It is much more difficult for institutions to measure impact in areas like community engagement or contribution to the profession, but I would really like to see some attempts. I certainly don't have a solution, just an interest in the problem. It is definitely an area I will be keeping an eye on.