Tuesday, March 27, 2018

How to Populate Quality Tab in ML Model Inspect page in Oracle Analytics Cloud

In this blog post we will discuss about how to Populate Quality Tab of a Machine Learning Model's Inspect page in OAC.

Assessing quality of a Machine Learning model is an important step in evaluating its performance. Various metrics like RSE, RAE Residuals , R-Squared Adjusted etc help in assessing the quality of the model prediction. If the error metrics are not satisfactory or does not meet user's goals, he/she can tune the model model further till the required level of accuracy is reached. So it is important to expose this quality information of a model in an intuitive and comprehensive fashion so that users can take next course of actions as necessary. Quality tab in Inspect page of ML model in Oracle Analytics Cloud aims to visualize and give complete information on model accuracy details. Here is a snapshot of Quality tab for Linear Regression Model that predicts Bike Rental Count:

In this blog we will talk about how to populate this Quality tab for a custom model. Quality Tab is populated by adding required Related datasets in Train Model script. More details on how to add a related dataset can be found in this blog. Quality tab is auto-populated if user adds the following related datasets:

For Numeric Prediction: If Residuals and Statistics datasets are added as Related datasets in Train Model script, ML framework in OAC takes information from that dataset and populates the Quality tab. Graph is populated using Residuals dataset and all the error metrics are populated using Statistics Dataset. Image shown above shows how Quality tab of a Numeric Prediction model looks like. Here is a sample code that shows how to add Statistics and Residuals Related datasets:

        #residuals dataset
        residuals_mappings = None
        residuals_ds = ModelDataset("Residuals", residuals_df, residuals_mappings)

        # statistics dataset
        statistics_mappings = None
        statistics_ds = ModelDataset("Statistics", statistics_df, statistics_mappings)

For Classification: For Binary and Multi-Classification models Confusion Matrix and Statistics Related datasets are used to populate the Quality tab. Here is a sample snapshot of Quality tab for Classification models:

Confusion Matrix Related dataset is used to populate the Confusion matrix table that can be seen in the image and Statistics Related dataset is used to populate the metrics. Here is a sample code that shows how to add Confusion Matrix and Statistics related datasets:
            metrics['Statistics'] = stats
            metrics['Confusion Matrix'] = confMatrix
            self.add_datasets(metrics=metrics, model=model)

Quality tab is updated automatically if user changes the values of user or model tuning parameters and trains the model. 

Related Blogs: How to build Train/Apply Model Custom Scripts in OAC, How to create Related DatasetsHow to use inbuilt methods in OAC to Prepare data for Training/Applying ML Model 


calfre 010 said...

Your blog is so nice, and the article is very good it helps to so many people.
Oracle Financials Training in Ameerpet, Hyderabad

meena resma said...

Oracle SOA Online Training

Oracle SOA Online Training Bangalore

Coepd BA Trainings said...

We at COEPD provides the best web Designing course in Hyderabad. Your search to learn web designing technology ends here at COEPD. Here, we are an established training institute who have trained more than 10,000 participants in all streams. We will help you to convert your passion to learn into an enriched learning process. At COEPD, we provide finest web designing technology covering Photoshop, HTML, CSS, HTML5, CSS3, JavaScript, Boot Strap, and JQuery. We train participants to be solution providers and creative engineers.


kritika mytectra said...

myTectra the Market Leader in Machine Learning Training in Bangalore
myTectra offers Machine Learning Training in Bangalore using Class Room. myTectra offers Live Online Machine Learning Training Globally. Read More

Sangeetha Sathyan said...

I am very happy to visit your blog. This is definitely helpful to me, eagerly waiting for more updates.
ccna Training in Chennai
ccna institute in Chennai
R Programming Training in Chennai
Python Classes in Chennai
Best Python Training in Chennai
CCNA Training in Anna Nagar
CCNA Training in Porur

jenifer irene said...

It’s really a Great Post .Looking for Some More Stuff
Aviation Academy in Chennai
Air hostess training in Chennai
Airport management courses in Chennai
Ground staff training in Chennai

Post a Comment