Big Data Scoring with UPPI for IBM Pure Data (for Analytics and Hadoop)

In-database scoring is one of the most straightforward ways to gain insights from Big Data. It is no surprise then that the Zementis Universal PMML Plug-in (UPPI) is now being offered for a variety of database platforms. These include IBM Pure Data for Analytics (Netezza), Pivotal/Greenplum, SAP Sybase IQ, Teradata and Teradata Aster. Zementis also offers UPPI for Hadoop/Hive, including IBM Pure Data for Hadoop as well as InfoSphere BigInsights. It is in this context that we travelled to Vegas to attend the IBM Information on Demand (IOD) Conference.

I must say, I am always impressed by the IBM universe of products and tools that are being offered for analytics (descriptive and predictive) as well as Big Data in general. Zementis had a booth inside the Pure Data exhibit area and next to all the Pure Data appliances. As you can imagine, traffic was solid not just because of all the blinking lights but also because the conference itself attracts a lot of people. I believe there were 14 thousand attendants this year.

Why in-database scoring? Well, simple. Not all analytic tasks are born the same. If one is confronted with massive volumes of data that need to be scored on a regular basis, in-database scoring sounds like the logical thing to do. In all likelihood, the data in this case is already stored in a database and, with in-database scoring, there is no data movement. Data and models reside together hence scores and predictions flow on an accelerated pace.

Why scoring in Hadoop? Big Data and Hadoop are somewhat synonymous terms these days, since the latter offers an important technological platform to tackle the challenge of analyzing large volumes of data. In fact, predictive analytics is paramount for companies to extract value and insight from such data. By offering the Universal PMML Plug-in (UPPI) for Hadoop, Zementis takes a big step in making its technology available for companies around the globe to easily deploy, execute, and integrate scalable standards-based predictive analytics on a massive parallel scale through the use of Hive, a data warehouse system for Hadoop.

UPPI brings together essential technologies, offering the best combination of open standards and scalability for the application of predictive analytics. It fully supports the Predictive Model Markup Language (PMML), the de facto standard for data mining applications, which enables the integration of predictive models from IBM/SPSS, SAS, R, and many more.

Making the case for PMML and ADAPA

If you are not familiar with PMML, the Predictive Model Markup Language, you may be wondering what all the fuss is about ...

PMML is the de facto standard to represent data mining and predictive analytic solutions. With PMML, one can easily share a predictive solution among PMML-compliant applications and systems  For example, you can build your model in R, export it in PMML, and use ADAPA, the Zementis Scoring Engine, to deploy it in production.

Many data mining models are a one-time affair. You use historical data to build the model and use it to analyze ... historical data. Wait! That sounds more like descriptive analytics, not predictive analytics. Well, that is sort of true. To be truly predictive, a data mining model needs to be applied to new data. These are the models that need to be operationally deployed and, from my point of view, these are the solutions that are truly revolutionizing the way we do business and live in the Big Data world.

If you want then to use your data mining model to make predictions when presented with new data, it needs to be a dynamic asset. It cannot be static. You need to be able to build it and instantly put it to use. And, that's where PMML and ADAPA come in handy.

Obviously, a few data mining tools try to lock you in. You happily build the model using tool A, just to realize that you need the same tool to execute it. In this case, you are missing out. Here are some of the benefits of moving your predictive model to ADAPA:

• Overcome speed/memory limitations
• Dramatically lower your infrastructure cost
• Tap into all the advantages of cloud computing with ADAPA on the Cloud (IBM SmartCloud or Amazon EC2)
• Produce scores in real-time (using Web Services or Java API), on-demand, or batch-mode
• Execute your models directly from Excel, by using the ADAPA Add-in for Excel
• Benefit from using a set of PMML-compliant model development tools (best of breed)
• Deploy your models in minutes
• Manage models via Web Services or a Web console
• Upload one or many models into ADAPA at once
• Benefit from the seamless integration of business rules and predictive models (yes, for those who need it, ADAPA comes with a business rules engine)

PMML and ADAPA allow you to use best of breed tools (not the same old tool) for the job at hand. Also, you can leverage the expertise from a diverse group of data scientists. That means, not all your data scientists need to be experts on a single tool. They can use different tools that share one thing in common, the PMML standard. And, once represented in PMML, models can be easily understood by all team members. PMML allows for transparency and, in doing so, fosters best practices.

Why not benefit from: 1) an open standard to represent data mining models; and 2) a proven scoring engine that consumes any version of PMML and make it available for execution right away, in real-time?

Keep also in mind that ADAPA's sister product, the Universal PMML Plug-in (UPPI), allows you to move the same PMML file in-database or Hadoop. UPPI is currently available for EMC Greenplum, SAP Sybase IQ, IBM Netezza, and Teradata/Aster. With UPPI for in-database scoring, there is no need to move your data outside the database. Data and models reside inside it and so there is minimal data movement and maximum scoring speed. UPPI is also available for Datameer and will soon be available for Hadoop/Hive.

Making a model operational in minutes has never been easier! And, it is all because of PMML and scoring tools such as ADAPA and UPPI.

Zementis is proud to announce PMML 4.1 support

PMML 4.1, the latest version of the Predictive Model Markup Language, is loaded with new and powerful features. 

Zementis is proud to announce support for PMML 4.1 throughout its scoring products, including:

• ADAPA on Site
• ADAPA on the Cloud (Amazon and IBM SmartCloud)
• UPPI for in-database scoring (IBM Netezza, SAP Sybase IQ, EMC Greenplum) 
• UPPI for Hadoop (Datameer).

We have also updated our PMML conversion process so that it now converts PMML files from older versions to version 4.1. In this way, every time a PMML file is presented to ADAPA or UPPI, it is automatically converted to PMML 4.1.

  

Our support for PMML 4.1 includes:

1) Scorecards (including reason or adverse codes and point allocation for complex attributes)

2) Post-processing: you can now transform scores into business decisions as well as output generic data manipulation steps

3) Multiple Models: a powerful and yet simpler way for the expression of model segmentation, composition, chaining and ensemble, which includes Random Forest models

4) Is the model scorable? The "isScorable" flag was added as a way to flag models not destined for production deployment, but that are nonetheless an important part of the model building cycle

5) New built-in functions (for pre- and post-processing).

With this new release and version update, ADAPA and UPPI can be used not only for deployment and execution of predictive solutions, but also for data analysis and processing before model training.

  

If you have any questions about PMML 4.1 and all the features supported in our products, please make sure to contact us or feel free to check out our PMML 4.1 forum for detailed support information.

Learn how the IBM / Zementis Partnership simplifies Predictive Analytics

Benefiting from Interoperability

  

PMML, the Predictive Model Markup Language, has become the de-facto standard to represent not only predictive models, but also data pre- and post-processing. In so doing, it allows for the interchange of models among different tools and environments, avoiding proprietary issues and incompatibilities. 

Model Building: IBM SPSS

IBM SPSS Modeler and IBM SPSS Statistics are extremely powerful data analysis and model building environments. This power is backed-up by their support of PMML. In either tool, predictive models as well as data transformations can be easily exported into PMML. IBM SPSS Statistics, for example, allows for automatic data preparation which can be exported into PMML and subsequently merged into the final PMML file for the entire solution. 

View on-demand replay of the joint IBM SPSS/Zementis webcast focusing on the synergies between IBM SPSS and Zementis ADAPA (presented, May 14th, 2012).  

Discover the benefits of executing your IBM SPSS models in ADAPA

Model Execution: ADAPA on the IBM SmartCloud

Once exported in PMML, your IBM SPSS models can be readily deployed in the Zementis ADAPA Scoring Engine, where they can be put to work immediately. To minimize total cost of ownership, model execution in ADAPA is now available as a service through the IBM SmartCloud. 

View on-demand replay of the joint IBM/Zementis webcast focusing on predictive analytics deployment and execution on the IBM SmartCloud (presented, May 24th, 2012).

Review IBM developerWorks article about executing predictive solutions using ADAPA on the IBM SmartCloud.

Discover features and capabilities of ADAPA on SmartCloud

In-database Scoring: UPPI for IBM Netezza 

Predictive solutions expressed in PMML can also be put to work inside the database with the Zementis Universal PMML Plug-in (UPPI) which is now available for IBM Netezza. Since UPPI transforms your complex predictive solutions into SQL functions, these can be readily used in any query and generate instant business decisions and insights where and when you need them.

Review the UPPI for IBM Netezza Product Data Sheet

Discover all the features and UPPI supported databases

Explore the IBM Netezza Analytics website