IBM’s Watson is one of the most appealing new technologies of the 21st century, and the most prominent example of the new category of “cognitive computing.” It burst upon the scene with a dramatic Jeopardy! win in 2011, and has now been adopted by a variety of business and health care organizations since then.

For several months I have been speaking with the firms and organizations that signed on to Watson deals at a relatively early stage. Since most of the earliest adopters of the technology were health care organizations, I’ll focus on that industry in this column.

The organizations I interviewed include Memorial Sloan Kettering Cancer Center (MSKCC) and MD Anderson Cancer Center (MDACC), Mayo Clinic, Welltok, (developer of the CaféWell Health Optimization Platform), LifeLearn (a Canada-based veterinary products and services company), and a health insurance company that chose to remain anonymous. These organizations have different official relationships with Watson (for example, MSKCC is a partner/co-developer, MDACC is a customer, and LifeLearn and Welltok are Watson ecosystem partners), but it’s safe to say that all of these early adopters were partners in the sense of providing critical domain knowledge to early Watson implementations.

WATSON IS GROUNDBREAKING

The people I interviewed (most of whom were overseers of their organization’s Watson projects) were uniformly enthusiastic about the technology—in some cases even after several years working with it. Dr. Mark Kris, the MSKCC oncologist who has led that institution’s Watson project since 2012, commented that, “It’s been a lot more complex, and taken a lot more time, than we had thought. But this is the way medicine is going to be practiced.” Jeff Margolis, the chairman and CEO of Welltok, says about the power behind his company’s “CaféWell Concierge app powered by Watson,” that, “Watson learns quickly from the corpus [body of knowledge that informs its recommendations] and doesn’t forget. It has spatial awareness and temporal understanding. It’s an amazing technology.” Dr. Steve Alberts, who is leading a Watson project at the Mayo Clinic that matches patients with clinical trials, said that, “It’s amazing how much unstructured and structured knowledge Watson can pick up on.”

THE “MOON SHOTS” DON’T HAPPEN QUICKLY

Several of the early Watson implementations were “moon shots—highly ambitious and complex projects that would be difficult to accomplish using any technology. MSKCC’s attempt to train Watson to know how to treat lung cancer, and MDACC’s work to help improve the quality of care for cancer patients with no access to cancer specialists (starting with a solution for acute leukemia), certainly qualify for this term. MDACC actually referred to its project as a moon shot, as its goal was to build a virtual expert called MD Anderson Oncology Expert Advisor© (or OEA© ) that is trained to not only support guideline-based and expert-recommended therapy decisions, but also share MD Anderson specialist’s experience in managing a specific type of cancer patient to maximize treatment benefits and improve outcome. In other words, it aims to not only share the clinical evidence but also the “art” of cancer care.

It’s not surprising that these difficult projects have taken a while to implement. MSKCC started working on its project in February of 2012, and it’s still going (although IBM has already made Watson Oncology Advisor based on MSKCC’s training available to hospitals in Thailand and India). In a way such projects will never be finished, as new knowledge and published content on cancer is always being published. Watson can continue to learn over time, which makes it a good technology for this purpose. However, some organizations may quake a bit at undertaking similar moon shots. There are, of course, less ambitious Watson projects that have been completed more quickly, such as the Welltok one, and many others outside of health care. But ambition takes time. Mike Rhodin, the head of IBM’s Watson business unit, said in an interview that, “We have worked on cancer platforms now with leading partners for a few years. People ask me why it takes Watson a few years to learn oncology. But I ask them how long does it take a human to learn it? The oncology leaders we are working with have spent decades learning what they know, so a few years for Watson seems reasonable.”

IT’S NOT JUST ABOUT THE TECHNOLOGY

In the early stages of these Watson implementations, the focus is, of course, on making the amazing technology work. But that is only the beginning of the organizational and business changes necessary for cognitive technologies to support substantial transformation. As Dr. Lynda Chin, who led the Watson-based project at MDACC, put it in an email, “An application like OEA© cannot deliver on its intended impact of improving patient outcomes worldwide without addressing the necessary network infrastructure, security and regulatory controls, data sharing/access/use contracts, and reimbursement, not to mention the culture of medicine and clinical adoption. Only through addressing these non-technical challenges, we will be able to translate a piece of technology, like OEA©, into impact. That is what separates an innovation from a transformation…that is what makes it a moon shot.”

EARLY ADOPTERS PAY AN ADAPTATION PRICE AND HOPE TO REAP SOME REWARDS

There is a price to be an early adopter of any technology, and Watson is no exception. In this case, the system has to be adapted to each new industry and knowledge domain in which it’s employed. It had to learn the structures of knowledge in oncology, veterinary medicine, wellness, and so forth in the cases I have researched. Dr. Alberts at the Mayo Clinic noted that Watson requires the knowledge it ingests to be unambiguous: “Our researchers had an intuitive understanding of clinical trials. But when the IBM engineers were putting the knowledge into Watson, they had to ask for many clarifications and clearer wording. We realized our eligibility criteria were sometimes a bit ambiguous.” Clarifying all this takes a while, of course. But the early adopters hope to reap some rewards, either monetary or reputational. Certainly it will be easier for later adopters to employ Watson in these fields.

WATSON NEEDS TO BE TRAINED

A related early adopter issue is that Watson needs to be trained. Dr. Mark Kris at MSKCC, and Dr. Chin at MDACC, said that training Watson is not unlike training a human medical student. Dr. Kris commented that, “It’s an apprenticeship form of training that takes years—there are lots of subtleties that Watson has to learn.” At MD Anderson, Watson programmers shadowed leukemia clinicians to understand how they talk and think. In the veterinary context, James Carroll, the CEO of LifeLearn, said that, “We had to start the process with teaching Watson what a dog was.” But training Watson to be a vet seems to be a bit faster than creating an artificially intelligent medical student; Carroll notes: “In about five months, we’ve taught Watson the approximate curriculum equivalent of a 4 year veterinary medical degree. But we’re not stopping there.” Again, the training costs are clearly higher for those who go first in a particular field.

THE CORPUS CAN BE PROBLEMATIC

One of Watson’s most well-known capabilities involves ingesting and digesting a “corpus”—a body of textual knowledge from which it can learn to make inferences. In areas where there is a well-established body of written knowledge, this can be pretty straightforward. Some vendors of information services — LifeLearn, for example, works with Wiley and Elsevier, and Thomson Reuters is also supplying online content to Watson — are making this ingestion process much easier. But the problem comes when the needed knowledge isn’t in the corpus. Dr. Kris at MSKCC comments: “We had three drugs approved in lung cancer this year. None of them are in the literature yet. And definitions of cancer and its variations are being redefined all the time as we understand the biological characteristics of each one. The science is changing more rapidly than the published literature.” So it’s not just a matter of feeding Watson some text. MSKCC and several other early adopters have had to employ human experts to create many question/answer pairs. At LifeLearn, for example, they’ve involved 70 veterinarians and vet technicians to create over 55,000 question/answer pairs. At MDACC, the oncology solution is designed to incorporate “expert recommendations” to address this gap between the speed knowledge advances and the time it takes to be codified into online guidelines.

WATSON ITSELF HAS BECOME FAR MORE COMPLEX AND CAPABLE

When these early adopters decided to embark upon their Watson journeys, there was only one type of Watson—the Q&A application that won Jeopardy! But now that system is one of about 32 different “cognitive APIs” (application program interfaces) offered under the Watson banner, and more are on the way. When I asked these early adopters whether they had plans to implement additional APIs, most intended to use only a few (MDACC says they are using ten or more). A couple of organizations are particularly interested in English language dialog. MSKCC is interested in image analysis, which would be of obvious value in oncology. IBM says they make available solution architects who can help customers figure out what APIs they need. One of the first commercial Watson implementations, a treatment preapproval application at a health insurance company, was not well suited to the Q&A application. Without some skilled architects we will probably see more mistakes in this regard.

THE PURPOSE ISN’T AUTOMATION

None of the early Watson adopters I spoke with seemed to think that their applications would put anybody out of work. The human cancer applications at MSKCC and MDACC will advise, but not replace, oncologists. At the Mayo Clinic, Watson will speed up the process of clinical trials matching and make clinicians and study coordinators more productive and effective at the process, but it won’t eliminate their jobs. LifeLearn’s Sofie will play the same role for veterinarians, augmenting their existing training and expertise. Welltok’s CaféWell Concierge application will support consumers 24/7, but won’t replace existing roles. This is good news, of course, from the standpoint of human experts. It does raise some questions about where the economic value will come from in these early Watson applications. Let’s hope that medical providers and payers and consumers are willing to pay for better advice.

Dr. Mark Kris at Memorial Sloan Kettering summed up the experience of early Watson adopters: “This is real and it’s going to revolutionize cancer and other types of medical care. It’s enormously complex, and we will never be finished until all cancers are cured. It’s clearly going to take some time to work out. After all, it took IBM three years to beat Jeopardy! But I don’t doubt at all that it will provide enormous value.” IBM’s Mr. Rhodin admits that, “These cancer and health care projects are on the far end of complexity. But when we started to apply Watson to problems beyond Jeopardy!, we thought IBM could make a difference in helping to solve an important global problem, and we wouldn’t do anything different today.”

Originally published in WSJ’s CIO Journal.