Productivity – How does Knovel help our customers measure it?

How do our customers justify their annual investment in Knovel or, for that matter, in any “productivity” tool? In every instance they think carefully about whether there are significant improvements in productivity, cost savings, and what their alternatives might be to get the same result. Engineering Managers and Information Professionals (the “decision makers” or “buyers”) tell us they make investments based on business cases showing how users (engineers) can save time by improving their productivity (reduce research time, accelerate time to market, faster processes, better technical service, more efficient warranty work, more cost effective designs…), providing reliable and relevant data (e.g., Google delivers too many spurious results), and cutting costs (e.g., dropping inefficient services, reduced “one off” content purchases…). These decisions are important as the dollars are often in the hundreds of thousands and sometimes millions.

 

A ‘trial” is widely accepted as the method for assessing the utility of a productivity tool, but measuring productivity is easier said than done. The conflict comes in setting useful productivity metrics and getting enough consistent data to measure. Managers will often agree to a “trial” for new software or an information service and find data from the trial is not gathered systematically and is qualitative and inconsistent (e.g., “the software is great and I think it can save me some time”). We help managers assess productivity by setting up “structured trials.” In addition, we have gathered a lot of data about the research habits of users from surveying thousands of engineers at corporations that have trialed Knovel. Using a well defined and repeatable process to assess productivity and historic data to set metrics goes a long way toward objectively measuring progress. The keys to which are:

 

• Setting buyer and user expectations before a trial begins
• Measuring productivity improvements quantitatively and qualitatively
• Careful design of survey questions that align with customer objectives
• A team with experience in trial process, survey design, and data analysis.

For instance, some of the measurements we might use include the level of survey participation, user estimates of time saved, and user’s views on the quality of their sessions. The real “art” is in crafting questions that yield a consistent and statistically significant set of measurable results. After a trial we work with our customers to analyze the data and compare to the agreed metrics to produce a report that will help guide their investment decision.

 

Traditionally we have focused this work on new customers but it is also important to gauge productivity gains over time and we are now beginning to survey long time users to gauge metrics and continuously assess the productivity benefit.

 

Thank-you,

Chris Forbes

CEO

Knovel