Azure Machine Learning Connector 1.0.0.0
The Azure Machine Learning Connector lets you run Azure ML pipelines from within RunMyJobs. You can specify the Azure ML pipeline you want by ID, or you can specify a pipeline endpoint. If you use one of the template definitions, you can supply parameter information in JSON format.
Prerequisites
- RunMyJobs 9.2.9 or later.
- Connection component 1.0.0.4 or later. Note that the Connections component will be installed or updated automatically if necessary when you install this extension.
- Azure Connections 1.0.0.3 or later.
- Privileges Required to Use Azure Connections.
- Azure Connections Extension (automatically installed by the Catalog when you install the Azure Machine Learning Connector).
- The Azure subscription ID and resource group name for the workspace where you are working with Azure ML.
- A Process Server with the ServiceForRedwood_AzureML service. For more information, see Connectors and Process Servers.
Contents of the Component
The Azure Machine Learning Connector consists of the following objects:
| Object Type | Name |
|---|---|
| Application | GLOBAL.Redwood.REDWOOD.AzureML |
| Process Definition | REDWOOD.Redwood_Azure_ML_SubmitPipeline |
| Process Definition | REDWOOD.Redwood_Azure_ML_SubmitPipelineEndpoint |
| Process Definition | REDWOOD.Redwood_Azure_ML_SubmitPipelineEndpoint_Template |
| Process Definition | REDWOOD.Redwood_Azure_ML_SubmitPipeline_Template |
| Process Definition Type | REDWOOD.Redwood_AzureML |
| Library | REDWOOD.Redwood_AzureML |
Redwood_Azure_ML_SubmitPipeline
This Process Definition lets you submit an Azure ML published pipeline by specifying its pipeline ID.
Parameters
| Tab | Name | Description | Documentation | Data Type | Direction |
|---|---|---|---|---|---|
| Parameters | connection
|
Connection | Specifies the connection for Azure ML. | String | In |
| Parameters | resourceGroupName
|
Azure Resource Group Name | The name of the Azure Resource Group that contains the desired Azure Pipelines. | String | In |
| Parameters | azureMLWorkspaceName
|
Workspace Name | The name of the Azure ML Workspace that contains the desired pipeline. | String | In |
| Parameters | pipelineId
|
Pipeline ID | The ID of the Published Pipeline to run. | String | In |
Redwood_Azure_ML_SubmitPipelineEndpoint
This Process Definition lets you submits an Azure ML published pipeline endpoint. If the endpoint contains more than one published pipeline, the default published pipeline is used.
Parameters
| Tab | Name | Description | Documentation | Data Type | Direction |
|---|---|---|---|---|---|
| Parameters | connection
|
Connection | Specifies the connection for Azure ML | String | In |
| Parameters | resourceGroupName
|
Azure Resource Group Name | The name of the Azure Resource Group that contains the desired Azure Pipelines. | String | In |
| Parameters | azureMLWorkspaceName
|
Workspace Name | The name of the Azure ML Workspace that contains the desired pipeline. | String | In |
| Parameters | endpointName
|
Pipeline Endpoint Name | The name of the Pipeline Endpoint to run. This will submit the Default Pipeline for that endpoint. | String | In |
Redwood_Azure_ML_SubmitPipeline_Template
Template definition for submitting Azure ML published pipelines, optionally with custom parameters.
Parameters
| Tab | Name | Description | Documentation | Data Type | Direction |
|---|---|---|---|---|---|
| Parameters | connection
|
Connection | Specifies the connection for Azure ML | String | In |
| Parameters | resourceGroupName
|
Azure Resource Group Name | The name of the Azure Resource Group that contains the desired Azure Pipelines. | String | In |
| Parameters | azureMLWorkspaceName
|
Workspace Name | The name of the Azure ML Workspace that contains the desired pipeline. | String | In |
| Parameters | pipelineId
|
Pipeline ID | The ID of the Published Pipeline to run. | String | In |
| Pipeline Parameters | experimentName
|
Experiment Name | The name of the Experiment to run the pipeline in. | String | In |
| Pipeline Parameters | displayName
|
Display Name | The display name for the pipeline run. | String | In |
| Pipeline Parameters | description
|
Description | The description for the pipeline run. | String | In |
Redwood_Azure_ML_SubmitPipelineEndpoint_Template
Template definition for submitting Azure ML published pipeline endpoints, optionally with custom parameters.
Parameters
| Tab | Name | Description | Documentation | Data Type | Direction |
|---|---|---|---|---|---|
| Parameters | connection
|
Connection | Specifies the connection for Azure ML | String | In |
| Parameters | resourceGroupName
|
Azure Resource Group Name | The name of the Azure Resource Group that contains the desired Azure Pipelines. | String | In |
| Parameters | azureMLWorkspaceName
|
Workspace Name | The name of the Azure ML Workspace that contains the desired pipeline. | String | In |
| Parameters | endpointName
|
Pipeline Endpoint Name | The name of the Pipeline Endpoint to run. This will submit the Default Pipeline for that endpoint. | String | In |
| Pipeline Parameters | experimentName
|
Experiment Name | The name of the Experiment to run the pipeline in. | String | In |
| Pipeline Parameters | displayName
|
Display Name | The display name for the pipeline run. | String | In |
| Pipeline Parameters | description
|
Description | The description for the pipeline run. | String | In |
Setup
To install the Azure Machine Learning Connector and create a connection to Azure AD:
-
Locate the Azure Machine Learning Connector component in the Catalog and install it.
- Set up an Azure subscription and an appropriate resource group in the Azure Connections Extension.
Submitting an Azure Machine Learning Pipeline
To submit an Azure Machine Learning pipeline:
-
Submit the Redwood_Azure_ML_SubmitPipeline Process Definition.
-
Choose your Azure AD connection from the Connection dropdown list.
-
Choose the desired resource group name from the Azure Resource Group Name dropdown list. If no options are visible here, you need to add an entry in the Azure Subscription user interface to register the subscription ID and resource group that your Azure ML workspace resides in.
-
Enter the workspace name in the Workspace Name field.
-
Enter the ID of the pipeline in the Pipeline ID field.
-
Click Submit.
Submitting an Azure Machine Learning Pipeline with a Template
To create a customized submit pipeline Process Definition, optionally with default values:
-
Right-click the Redwood_AzureML_SubmitPipeline_Template Process Definition and choose New (from Template) from the context menu. The New Process Definition pop-up window displays.
-
In the Definition tab, delete the default Application value (if any) and substitute your own Application name if desired.
-
In the Parameters tab, enter any Default Expressions you want to use.
-
When specifying the Connection value, use the format
EXTCONNECTION:<partition>.<connection name>. -
Optionally specify a default Azure resource group name, workspace name, and pipeline ID.
-
If you want to add Pipeline Parameters, put them in a parameter group named
Pipeline Parameters, as shown in the following screen shot.
-
-
In the Definition tab, use JSON to specify any parameters you want to pass to the pipeline. You can enter these manually or use dynamic parameters from the Pipeline Parameters group. Sample JSON is provided, but you will most likely need to customize it for your particular use case. For information about the specific parameters available, see Viewing Pipeline Endpoint Documentation in Azure.
-
Save the new Process Definition.
Submitting an Azure Machine Learning Pipeline Endpoint
To submit an Azure Machine Learning pipeline endpoint:
-
Submit the Redwood_Azure_ML_SubmitPipelineEndpoint Process Definition.
-
Choose your Azure AZ connection from the Connection dropdown list.
-
Choose the desired resource group name from the Azure Resource Group Name dropdown list. If no options are visible here, you need to add an entry in the Azure Subscription user interface to register the subscription ID and resource group that your Azure ML workspace resides in.
-
Enter the workspace name in the Workspace Name field.
-
Enter the name of the pipeline endpoint in the Pipeline Endpoint field. Note that if this endpoint has multiple pipelines, the default pipeline will be used.
-
Click Submit.
Submitting an Azure Machine Learning Pipeline Endpoint with a Template
To create a customized submit pipeline endpoint Process Definition, optionally with default values:
-
Right-click the Redwood_Azure_ML_SubmitPipelineEndpoint_Template Process Definition and choose New (from Template) from the context menu. The New Process Definition pop-up window displays.
-
In the Definition tab, delete the default Application value (if any) and substitute your own Application name if desired.
-
In the Parameters tab, enter any Default Expressions you want to use.
-
When specifying the Connection value, use the format
EXTCONNECTION:<partition>.<connection name>. -
Optionally specify a default Azure resource group name, workspace name, and pipeline endpoint name.
-
If you want to add Pipeline Parameters, put them in a parameter group named
Pipeline Parameters, as shown in the following screen shot.
-
-
In the Definition tab, use JSON to specify any parameters you want to pass to the pipeline. You can enter these manually or use dynamic parameters from the Pipeline Parameters group. Sample JSON is provided, but you will most likely need to customize it for your particular use case. For information about the specific parameters available, see Viewing Pipeline Endpoint Documentation in Azure.
-
Save the new Process Definition.
Viewing Pipeline Endpoint Documentation in Azure
To view the schema for a pipeline endpoint in Azure:
-
Go to Azure Machine Learning Studio.
-
Click Pipelines on the left.
-
Click the Pipeline endpoints tab.
-
Click the pipeline you want to run.
-
Under Pipeline endpoint overview, click the REST endpoint documentation link.