An Honest Look at the State of Big Data in Insurance

Jeff Goldberg

With the recent publication of Novarica’s Analytics and Big Data at Insurers report, it’s time to take an honest look at the state of big data in the industry. One of the most telling–and disappointing–charts showed that of the insurers working with big data, seventy percent were using traditional computing, storage, database, and analytics technology, meaning they’re working with SQL databases in their existing environment. Of all the other technology options (Hadoop, NoSQL, columnar databases, etc) only a small percentage of insurers use them and almost no insurer uses them extensively.


Compare that to the percentage of insurers who say they are using big data sources, which is significantly higher than the percentage of insurers using big data technology. This includes third-party consumer or business data, geospatial data, weather data at the top the list, with audio/video data, telematics, social-media content, and internet clickstreams lagging behind. But what’s really happening when those big data sources are loaded into a traditional structured database? Most likely, a few key elements are pulled from the data and the rest is ignored, allowing the so-called big data to be treated like “small data,” processed along with all the policy, claims, and customer data already being stored. For example, while a weather feed might be coming with minute-by-minute updates, most insurers are probably just pulling region condition along with daily temperature highs and lows, filtering it down to a subset that stores easily. While I’m not saying such reduced data doesn’t augment an insurer’s understanding of incoming claims (for example), it’s far from what we think about when we imagine how a big data analysis of the weather might impact our understanding of insurance claim patterns.

There’s no denying that there are a few exciting use cases of insurers truly leveraging big data, but it’s still extremely limited. The industry as a whole is getting much better at data analysis and predictive modeling in general, where the business benefits are very clear. But the use cases for true big data analysis are still ambiguous at best. Part of the allure of big data is that insurers will be able to discover new trends and new risk patterns when those sources are fully leveraged, but “new discoveries” is another way of saying “we’re not yet sure what those discoveries will be.” And for many insurers, that’s not a compelling enough rationale for investing in an unfamiliar area.

And that investment is the second problem. The biggest insurers may have the budget to hire and train IT staff to work on building out a Hadoop cluster or set up several MongoDB servers, but small to mid-size insurers are already stretched to their limits. Even insurers who dedicate a portion of IT budgets to innovation and exploration are focusing on more reliable areas.

What this means is that insurers–no matter how many surveys show they anticipate its adoption–will likely not see a significant increase in big data tech. However, that doesn’t mean the industry will let big data pass it by. Instead, much of the technology innovation around big data will need to come from the vendor community.

We’re already seeing a growing number of third-party vendors that provide tools and tech to do better analysis and get deeper understanding from big data, a second-generation of big data startups. Most of these vendors, however, expect that the insurer will already have a Hadoop cluster or big data servers in place, and (as we know) that’s exceedingly rare. Instead, vendors need to start thinking about offering insurers a “big data in a box” approach. This could means SaaS options that host big data in the cloud, appliances that offer both the analysis and the infrastructure, or even just a mix of professional services and software sales to build and manage the insurer’s big data environment on which the licensed software will run.

We’ll also begin to see insurance core system vendors begin to incorporate these technologies into their own offerings. The same thing has happened for traditional data analytics, with many top policy admin vendors acquiring or integrating with business intelligence and analysis tools. Eventually they’ll take a similar approach to big data.

And finally, some third-party vendors will move the entire big data process outside of the insurers entirely, instead selling them access to the results of the analysis. We’re already seeing vendors like Verisk and LexisNexis utilize their cross-industry databases to take on more and more of the task of risk and pricing assessment. Lookups like driver ratings, property risk, and experience-based underwriting scores will become as common as credit checks and vehicle history. These third-party players will be in a better position to gather and augment their existing business with big data sources, leveraging industry-wide information and not just a single book of business. This mean that smaller insurers can skip building out their own big data approach and instead get the benefits without the technology investment, and they can compete against bigger players even if their own store of data is relatively limited.

So while the numbers in Novarica’s latest survey may look low and the year-on-year trend may show slow growth, that doesn’t mean big data won’t transform the insurance industry. It just means that transformative change will come from many different directions.

On Tuesday, July 14th at 2 pm I’ll be hosting a Business Intelligence and Analytics webinar, which will review the recent report, go into more detail on big data, and cover how insurance is being transformed by the growth in available data and information both inside and outside the enterprise. For more information, visit:

Related Report

Top Stories Life/Annuity

Steven Kaye

We’ve just published our  Novarica Industry Intelligence Brief for Life and Annuity for June 2015. These reports highlight some of the most interesting industry stories from the past month, and present them along with Novarica commentary. Commentary is available to clients only, but we’ve posted direct links to the stories below:

  • ACI Specialty Benefits’ MacroLife uses gamification to encourage employee benefit interactions. Full Story.
  • Current financial planning models may not account sufficiently for unpredictability in portfolio returns and in investor spending due to life
    events. Full Story.
  • VSP is experimenting with embedding wearables in glass frames to track various health data. Full Story.
  • ADP has cut off its ADP RUN payroll clients from sharing data with Zenefits, claiming Zenefits has been pulling confidential data in an insecure fashion. ADP also filed a defamation lawsuit against Zenefits, claiming Zenefits was conducting a malicious PR campaign against the payroll services provider.Full Story here and here.
  • CB Insights reports close to a 500% increase in carriers’ investment in tech startups in 2014 globally. Full Story.
  • Aflac can now receive, process, and pay claims within one business day.Full Story.

For Novarica commentary, clients can download the Brief at

Top Stories Property/Casualty

Steven Kaye

We’ve just published our Novarica Industry Intelligence Brief for Property and Casualty for June 2015. These reports highlight some of the most interesting industry stories from the past month, and present them along with Novarica commentary. Commentary is available to clients only, but we’ve posted direct links to the stories below:

  • Allstate has filed a patent for a driver behavior database that might include driver health information. The carrier might sell the information to third parties, with policyholder permission.Full Story.
  • The FAA has allowed the Property Drone Consortium to operate drones on behalf of customers. Full Story.
  • American Family and Liberty Mutual are both offering insurance discounts to policyholders who agree to share data from their Nest Protect smoke detectors.Full Story
  • Allstate has a portable hail damage evaluation system that vehicles can be driven into, leveraging analytics software and cameras.
    Full Story
  • BOLT Solutions Inc. introduced Bolt Google Connect to help carriers participate in Google Compare.Full Story.
  • Marsh has published a report on insuring drones, addressing insurance capacity and regulations affecting the market. Full Story.
  • Munich Re issued a report on potential impacts of autonomous vehicles on commercial and personal insurers.Full Story
  • CB Insights reports close to a 500% increase in carriers’ investment in tech startups in 2014 globally.Full Story.
  • The International Association of Claims Professionals at a regional conference noted 3-D laser scanning and printing as an increasingly useful
    technology for recreating environments to identify risks and to better understand accidents. Full Story.
  • The 2016 Chevy Malibu enables parents to get in-vehicle reports on their teenagers’ driving, including distance drive, number of braking events,
    and speed, via the vehicle infotainment system. Full Story.
  • Erie Insurance has been testing Google Glass for use in claims adjusting. Full Story.
  • One forensic investigation firm is using robots for property investigations indoors, in the dark, or otherwise not suited for drones currently. Full Story.

For Novarica commentary, clients can download the Brief at:

What Do Blue Jeans and Business Intelligence Have in Common?

Rob McIsaac

Change is clearly all around us, with interesting new technology announcements seeming to arrive at a dizzying pace. From self-driving cars, to the Internet of Things, to the advent of new developments languages (e.g., Swift), finding the informational “gold” inside all of the noise can seem like a full time task. And, of course, missing key developments can become a bit of an occupational hazard. That said, two recent and relatively quiet developments are particularly interesting and with potential long term implications for insurance carriers and risk managers. Even the seemingly mundane can have important implications!

The first announcement came out of Google’s recent developer conference. Somewhat lost in all the technical announcements and mobile application updates was news that Google and Levi Strauss (yes, the denim folks) will be partnering on the development of “smart fabrics” that can become a component in the ecosystem evolving around wearable technology. With Baby Boomers aging and blue jean sales in decline, rather than hunker down and hope for the best, Levi’s has decided to break out in a new direction with what might at first seem an odd connection. Not odd at all, of course, when considering the need to do something different to appeal to a new generational cohort and when positioning the company to remain relevant in the face of changing consumer preferences. Insurance carriers should take note on both those elements purely from a business strategy formulation perspective.

More importantly, however, carriers across multiple lines of business should consider the product and service implications for the advent of a range of wearable technologies that will go significantly beyond the current crop of Fitbit and related activity based devices. Full wardrobes, including presumably my favorite 505 jeans that can become part of a two way communication of information could fundamentally alter customer expectations around how they will interact with financial institutions and others they choose to engage with economically. Smart / connected fabrics can have very real implications for both commercial and personal protection products and the way carriers understand (and influence) risk outcomes.

The other development speaks to the attraction of leading edge skills and the evolution of research that can have commercial application. Historically, a significant amount of primary research with leading technologies has been done in and around research universities. As technologies matured opportunities were created for patents to be secured by the universities, which in turn supported a vibrant community of startup entities which could take products to market. Red Hat, SUN and SAS are but three examples of this model which combined private and public monies to fuel innovation.

Increasingly, however, innovative companies are finding it to their advantage to go after talented people earlier in their careers, allowing them to combine private funding and talent to create early mover advantage for themselves. In some cases the transition has been quiet with companies luring students away from traditional graduate and post doc programs. In some cases it has been loud; Uber recently plucked a cadre of talent out of Carnegie-Mellon’s robotics program in order to accelerate (pun intended) their own efforts to move autonomous vehicles into the fast lane.

Insurance carriers aren’t immune to this phenomenon either. MassMutual recently announced the creation of a Big Data analytics group that will be housed near the UMass campus rather than at their home office. They also created a startup venture, Haven Life, to experiment with, among other things, using public data and analytics to sharply reduce underwriting cycle time for life insurance.

Getting access to high caliber talent early can create a notable competitive and cost advantage for companies rolling to step into this space.

Both of these events are reflective of significant changes underway in the broader economy today, where new products … and the human capital required to create them … can come together in some surprising, and surprisingly quick, ways. On the Google / Levi’s front, new products should create fascinating new IoT opportunities. Of course, the flood of Big Data that comes from this could be lost on entities still stuck on sorting out small data problems. That will then create new demand for attracting and retaining the talent needed to turn all that raw data into actionable information. Getting into the game earlier to groom and develop those capabilities could be a valuable strategic move for carriers. Conversely, failing to do so, could create real and unintended business system risks that could have dire long term consequences.

In the business intelligence and analytics space it is clearly “Game On” now. This will be interesting to watch as winners and losers emerge.

On Tuesday, July 14th at 2 pm (ET) Jeff Goldberg, VP of Research & Consulting will be hosting a Business Intelligence and Analytics webinar, which will cover how insurance is being transformed by the growth in available data and information both inside and outside the enterprise. For more information, visit:

Related Report

How Technology will Shape the Individual Annuity Marketplace in 2015 and Beyond

Mitch Wein

With the release of Novarica’s new Business & Technology Trends: Individual Annuity report, I wanted to highlight a few trends which our research uncovered in this blog.

First, 2015 is a pivotal year for annuities. It is the year when Millennials will equal Baby Boomers in the workplace. Once Baby Boomers start retiring over the next 15 years, they will go from 50% of the workforce to under 10%. Second, the impact of the Great Recession has run its course, with interest rates poised to rise in the 4th quarter of 2015 and beyond.

These events will drive huge demand for innovative annuity product features, such as alternative asset classes via funds and sub account investment options to guarantee income at retirement for baby boomers. In order to deliver, core systems must be replaced to support new product features and agent portals must be updated to support straight-through- processing to provide the operational efficiency required.

In addition to core system replacement and straight-through-processing some of the technology priorities noted in our new report include:

  • Use of social media in marketing tools
  • More sophisticated actuarial systems
  • Improved analytics and sales reporting
  • Proper tools for enhanced risk management, with a focus on internal controls

Novarica Insurance Core Systems Map Individual Annuity

Novarica Insurance Core Systems Map Individual Annuity

In order for insurers to survive and thrive in the individual annuity marketplace, technology will continue to play a vital role in terms of speed to market, customer acquisition and retention and product flexibility and pricing. For more information, download a free preview of our Business & Technology Trends: Individual Annuity report at: or email me to set up a complimentary 30 minute consultation.

Talent Quest: A Whole New IT Ballgame?

Rob McIsaac

Innovation and transformative thinking are more than just buzz phrases for insurance carrier IT organizations. In an effort to avoid their own Kodak (or Blockbuster) moments, CIO’s and their teams are increasingly focused on how technology can be leveraged to create real and meaningful differentiation for their carriers. It is both a valued undertaking and a legitimate concern if they fail. With many aspects of insurance now highly commoditized and new competitive threats emerging, just getting better at “the same old thing” won’t be good enough going forward. In fact, that may be a pathway to the proverbial “death by a thousand cuts”.

As a result, a rigorous governance and prioritization process becomes an early requirement for this new order of things. Can anyone deliver power better than the local public utility? Or email better than a legitimate cloud provider like Google or Microsoft? In both cases the answer is “no”, although hobbyist interest will persist.

For the future, real value will come in areas such as Analytics (turning data into actionable information) and mobility, to name but two of a long list of high priority events. For carriers that persist in Majoring in Minors, the future may represent a very tough row.

As the new priorities take hold, however, a new challenge will emerge: how to find and retain the best talent. The initial quest is critical of course. If you can’t find them, retention will take care of itself.

A critical new paradigm to consider is that carriers may need to take on a more proactive role in finding talent much earlier than was once considered “normal” in career management parlance. For example, for research scientists, the path to success historically went through research universities that were able to secure funding from large Federal agencies such as the NIH and NASA. Increasingly, however, the best path forward is through private industry where capital pools may provide a more stable and less political engine for keeping the research (and the patent machine) humming.

Uber’s recent move to hire a notable swath of the robotics lab away from Carnegie-Mellon University is a high profile example of the urgency some employers now approach the market for key human resources. At NC State University, students coming through graduate programs in Analytics are hired well before graduation. The competition is tough for people who have the right skill and drive and for employers waiting until after the commencement music ends, there may well be slim pickings. These are not phenomenon reserved just to the RTP or other “Silicon Valley Like” areas either. Getting into action early is crucial to both getting the right talent and moving organizational culture in the right direction.

MassMutual’s decision to set up an operation close to the UMass campus is a great industry example of a carrier moving aggressively to get the right skills cornered early and bring top talent in before they get siphoned off in another direction or to another industry. MetLife moving to RTP has put their IT group in close proximity to three major research universities that can provide a talent infusion of significant proportions.

We’ve seen this before, of course. For many professional sports teams, the talent pool they look to is younger than a generation ago … and as business enterprises they have benefitted. For carrier IT organizations considering the best way to move forward the watchword may well be “Get in the Game”.

Top Technology Priorities for Personal Lines Carriers

Jeff Goldberg

Today’s personal lines marketplace is more competitive than ever due to slow growth, intense price competition and customer acquisition costs rising.

Personal Lines insurers have always been a leader in insurance technology innovation and conversations with CIOs and research in the space shows that trend will continue, with technology playing an ever-larger role in insurers’ ability to attract, retain, and profitably serve clients. Across the industry, insurers continue to make investments across the Novarica Insurance Core Systems Map.

Novarica Insurance Core Systems Map: Personal Lines

Novarica Insurance Core Systems Map: Personal Lines

In a market with very competitive conditions and intense profitability pressures, personal lines carriers are focusing on growth strategies, expense reduction, and improving underwriting results. Below I have listed four technology priorities CIOs and business executives should consider to remain competitive.

Business Intelligence
A data quality initiative, which examines data warehousing, operational data stores, and appropriate data marts, is key before undertaking more advanced business analytics initiatives. Once data quality is ensured, carriers can then overlay business intelligence tools. Predictive analytics tools for carriers with sufficient data are becoming more popular. Small carriers should look at working with an organization that can provide pooled data and insights. All carriers can use models to improve underwriting insights, to more consistently apply pricing, and to improve claims activities. In addition, third party big-data sources are going to become more and more prevalent for personal lines insurers. Companies who take advantage of this first will have an edge in pricing and retaining business.

Policy Administration Systems
Upgrades to policy admin systems will help carriers gain operational efficiencies and flexibility in the ability to add data. Using business rules to manage workflow and predictive analytics to build pricing models can improve risk selection, risk pricing, and reduce operating expenses. Carriers should look for highly configurable solutions with product configurators, simple rules, and tools for launching new rating algorithms. They should also look for the ability for business units to make their own modifications, though practical experience with configurable systems reveals IT often still ends up managing most changes. As long as the time and cost of such work for IT is reduced, that’s still a big value.

Agent Connectivity
Extending functionality to the agents continues to rise in importance. It’s less and less about differentiation and more and more simply the price to pay to be in the game. At this point in time, most personal lines insurers have built an agent portal, and are often quite proud of the results. As a next step, both agents and carriers would prefer to receive and provide information electronically and process that information with as little human touch as possible, eliminating double entry. Real-time upload, download, and data translation deliver tangible benefits including reduced costs of handling, improved data quality, and improved turnaround time.

Claims Management
Streamlining claims management by automating processes improves customer service by speeding up claims service, providing consistent and fair best practices to all customers, and delivering personal service. On top of these customer benefits, insurers who have implemented modern claims systems report tangible speed-to-market benefits. If a carrier hasn’t already begun to upgrade their claims administration system, now is the time to start. Carriers who are using modern systems are rapidly gaining competitive advantages by improved efficiencies in claims handling and improved data leading to better outcome management. In addition, better claims processing has become a significant part of how personal lines insurers market themselves to consumers and how consumers select an insurer.

These technology priorities are based on the expertise of Novarica’s staff, conversations with members of the Novarica Insurance Technology Research Council, and a review of secondary published resources. For more information, download a free preview of our new Business and Technology Trends: Personal Lines report at: or email me to set up a complimentary 30 minute consultation.

Related Reports

Unexpected Competition: Only A Heartbeat Away!

Rob McIsaac

At Pegaworld this week, Dr. Mark Boxer, Global CIO at Cigna Insurance, delivered a powerful keynote address on the evolution of healthcare and the supporting technology in the United States. While acknowledging a range of challenges in the healthcare system today, he likened this to the opening lines from “A Tale of Two Cities”: it is The Best of Time and The Worst of Times, concurrently. The Best of Times component is characterized by the flood of new technology which is fundamentally allowing them to create and participate in a digital ecosystem that is like nothing we have ever seen before. The implications for other forms of insurance, including life carriers, are significant. The competitive game is about to get a whole lot more interesting.

The use of wearable technologies and related devices are anticipated to create a range of new connection points between customers and the carrier which allow for a far more interactive form of engagement. Through the use of these capabilities, Dr. Boxer talked about the transition from “sickness care” to “wellness care”, which allow them to refine how healthcare is integrated into consumer lives while making the process interactive. Monitoring of issues (e.g., diabetes, activity levels, etc.) is allowing them to create a lower cost model that has better outcomes. That’s a significant result that offers the promise of fundamentally re-architecting the nature of the relationship with their customers.

He also noted that their approach to client engagement starts at the point of “enrollment”. And, of course, this is the exact same enrollment process that Group Life carriers are participating in to gain mind, and wallet-share, from plan members. And therein lies a potentially significant challenge for traditional Group (and Individual) insurance carriers.

As we watch the transition from Group to Voluntary Benefits emerge, concurrently with demographic shifts that alter the face of tomorrow’s customer, there’s a clear wakeup call that is warranted. While many carriers continue to incrementally address issues related to aging technical environments in fairly traditional ways, considering like-structured companies as the future state competition, there are new threats that are emerging quickly. The Cigna depiction of how they anticipate weaving themselves into the daily lives of customers, where they are both receiving and sending information that is value added in both directions, is instructive. And, of course, armed with all the data and the digital relationship, the potential to cross sell other things to health insurance customers seems intuitively obvious. Being able to extend from traditional health coverages into other forms of coverage (e.g., life, disability, critical illness, etc.) seems a relatively modest extension of capabilities.

Which means that Group / Individual Life carriers, who anticipate that the voluntary benefits market will be a future state growth engine, may suddenly find themselves facing off against an unexpected … and unexpectedly well prepared … foe. The Internet of Things isn’t a theoretical construct or something that might impact insurance some day. It’s coming faster than many think. This will become very, very interesting!

As a result, the Tail of Two Cities metaphor could have a somewhat different meaning than Dr. Boxer intended. Armed with these capabilities, it could be the Best of Times for well prepared carriers … and the Worst of Times for those who fail to effectively plan for and integrate the “IoT” into their own operational fabric.

Creating an Enterprise Blueprint and Roadmap for Insurance Carriers

Mitch Wein

On our most recent webinar: “IT Strategy and Architecture” we talked about how the demands on insurance IT have never been higher. Insurer IT groups are tasked with delivering an ever-expanding set of capabilities, including digital transformation, faster speed-to-market, better distributor and customer service, and better analytical capabilities.

Unfortunately, too many insurers are dealing with the challenges of overly complex IT environments and brittle or redundant systems that cripple their ability to deliver business value rapidly or cost effectively.

A vision is needed to address the business transformation, while leveraging technology trends and leading industry solutions. From this, a roadmap of projects must be defined to implement that vision, while keeping the business running.

Creating an Enterprise Blueprint and RoadMap for Insurance Carriers

Creating an Enterprise Blueprint and RoadMap for Insurance Carriers

All too often there is a disconnect between business and IT, but having a common purpose and common understanding throughout IT and other business units enables true teamwork. Novarica’s new IT Strategy and Architecture consulting offering can help insurers protect their long term investments, while guarding against Business/IT disconnect.

For more information, please view our webinar recording or email me to set up a complimentary 30 consultation.

Top Stories in Life/Annuity for May 2015

Steven Kaye

We’ve just published our Novarica Industry Intelligence Brief for Life and Annuity for May 2015. These reports highlight some of the most interesting industry stories from the past month, and present them along with Novarica commentary. Commentary is available to clients only, but we’ve posted direct links to the stories below:

  • MassMutual is cutting the term life application process from weeks to minutes using third-party data and algorithms. Full Story.
  • The National Association of Retirement Plan Participants offers a ranking of plan providers based on how well they help employees save for retirement. Full Story.
  • Employers offering voluntary benefits recognize the importance of mobile enrollment, according to a LIMRA survey. Full Story.
  • John Hancock’s purchase of Guide Financial gives them AI and behavioral finance tools to help both advisors and their clients.Full Story.
  • Will employers see more lawsuits from employees over excessive fees charged for retirement plan investments? Full Story.
  • HR software provider Zenefits is threatening benefits brokers’ business – and the market is rewarding it handsomely. Full Story.
  • The SEC may require mutual fund providers to provide disclosures of the effect of rising interest rates on the bond holdings of investors. Full Story.
  • NAIFA is trying to block implementation of the Department of Labor’s latest fiduciary ruling, and there may be some hope. Full Story.
  • Legal & General America launched a website to educate consumers on the need for life insurance and its real costs. Full Story.
  • Behavioral finance is all the rage, and LIMRA used it to understand why consumers don’t buy life insurance and how they can be encouraged to do so. Full Story.

For Novarica commentary, clients can download the Brief at:


Previous Novarica Industry Intelligence Briefs for Life/Annuity