create-spark-connection

Description

Creates a new Connection.

Usage

oci goldengate connection create-spark-connection [OPTIONS]

Required Parameters

--artifact-location [complex type]

This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--authentication-type [text]

Authentication type for Spark. AES_RPC and SASL_ENCRYPTED securityProtocol values require authenticationType to be SHARED_SECRET. Use SHARED_SECRET only when the target cluster is configured with spark.authenticate=true.

--cluster-endpoint [text]

Spark master URL used for job submission. Required for both submitMode SUBMIT and REST. e.g.: ‘spark://spark-master.example.com:7077’ High availability can be configured with a comma-separated list, for example: ‘spark://h1.example.com:7077,h2.example.com:7077’

--compartment-id, -c [text]

The OCID of the compartment being referenced.

--display-name [text]

An object’s Display Name.

--security-protocol [text]

Security protocol for Spark submission traffic. PLAINTEXT uses an unencrypted channel, TLS uses TLS for the selected submission endpoint, AES_RPC uses Spark RPC AES encryption, and SASL_ENCRYPTED uses legacy SASL encryption. AES_RPC and SASL_ENCRYPTED require authenticationType to be SHARED_SECRET.

--submit-mode [text]

Selects the submission channel. SUBMIT uses the Spark master RPC endpoint. REST uses the Spark standalone REST submission gateway.

--technology-type [text]

The Spark technology type.

Optional Parameters

--auth-secret [text]

The shared secret used for Spark authentication. Applicable when authenticationType is SHARED_SECRET.

Deprecated: This field is deprecated and replaced by “authSecretSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--auth-secret-secret-id [text]

The OCID of the Secret that stores the Spark authentication shared secret. Applicable when authenticationType is SHARED_SECRET. Note: When provided, ‘authSecret’ field must not be provided.

--cluster-placement-group-id [text]

The OCID(/Content/General/Concepts/identifiers.htm) of the cluster placement group for the resource. Only applicable for multicloud subscriptions. The cluster placement group id must be provided when a multicloud subscription id is provided. Otherwise the cluster placement group must not be provided.

--defined-tags [complex type]

Tags defined for this resource. Each key is predefined and scoped to a namespace.

Example:

{"foo-namespace": {"bar-key": "value"}}

This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--description [text]

Metadata about this specific object.

--does-use-secret-ids [boolean]

Indicates that sensitive attributes are provided via Secrets.

Deprecated: This field is deprecated. Sensitive attributes should be provided using the corresponding Secret OCID attributes of the connection (for example, passwordSecretId) instead of plain-text attributes. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

When set to true, all sensitive information must be provided as OCI Vault secrets using the corresponding *SecretId attributes of the connection (for example, passwordSecretId). Plain-text sensitive attributes (for example, password) must not be used. This ensures that sensitive information remains stored and managed in the customer’s OCI Vault rather than by the GoldenGate service.

When set to false, sensitive information must be provided in the corresponding plain-text attributes (for example, password) rather than in secret OCID attributes. In this mode, the sensitive information is stored by the GoldenGate service. If vaultId and keyId are not specified, the GoldenGate service uses Oracle-managed encryption keys to encrypt the stored data.

If vaultId and keyId are provided, the specified customer-managed key is used.

--freeform-tags [complex type]

A simple key-value pair that is applied without any predefined name, type, or scope. Exists for cross-compatibility only.

Example:

{"bar-key": "value"}

This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--from-json [text]

Provide input to this command as a JSON document from a file using the file://path-to/file syntax.

The --generate-full-command-json-input option can be used to generate a sample json file to be used with this command option. The key names are pre-populated and match the command option names (converted to camelCase format, e.g. compartment-id –> compartmentId), while the values of the keys need to be populated by the user before using the sample file as an input to this command. For any command option that accepts multiple values, the value of the key can be a JSON array.

Options can still be provided on the command line. If an option exists in both the JSON document and the command line then the command line specified value will be used.

For examples on usage of this option, please see our “using CLI with advanced JSON options” link: https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliusing.htm#AdvancedJSONOptions

--key-id [text]

References the OCI Vault key in the OCI Vault identified by vaultId.

Deprecated: This field is deprecated for GoldenGate connections. Sensitive attributes should be provided using the corresponding Secret OCID attributes of the connection (for example, passwordSecretId) instead of plain-text attributes encrypted with vaultId and keyId. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

The GoldenGate service uses this key to encrypt sensitive information (for example, password) that is provided in plain-text connection attributes through the API. This field is applicable only when doesUseSecretIds is set to false. If both vaultId and keyId are provided, the GoldenGate service uses the specified customer-managed key to encrypt the sensitive data. If neither vaultId nor keyId is provided, the GoldenGate service uses Oracle-managed encryption keys.

--key-store [text]

The base64 encoded content of the client key store used for mutual TLS. Provide this only when the target cluster requires client certificate authentication. Applicable when securityProtocol is TLS.

Deprecated: This field is deprecated and replaced by “keyStoreSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--key-store-password [text]

Password for the client key store. Applicable when securityProtocol is TLS and a client key store is provided. Required when ‘keyStore’ is provided.

Deprecated: This field is deprecated and replaced by “keyStorePasswordSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--key-store-password-secret-id [text]

The OCID of the Secret that stores the client key store password. Applicable when securityProtocol is TLS and a client key store is provided. Required when ‘keyStoreSecretId’ is provided. Note: When provided, ‘keyStorePassword’ field must not be provided.

--key-store-secret-id [text]

The OCID of the Secret that stores the client key store content. Applicable when securityProtocol is TLS. Note: When provided, ‘keyStore’ field must not be provided.

--key-store-type [text]

Client key store format used for mutual TLS. Applicable when securityProtocol is TLS and a client key store is provided. Defaults to JKS when not provided. Provide this only when a client key store is provided.

--locks [complex type]

Locks associated with this resource.

This option is a JSON list with items of type AddResourceLockDetails. For documentation on AddResourceLockDetails please see our API reference: https://docs.oracle.com/en-us/iaas/api/#/en/goldengate/20200407/datatypes/AddResourceLockDetails. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--max-wait-seconds [integer]

The maximum time to wait for the work request to reach the state defined by --wait-for-state. Defaults to 1200 seconds.

--nsg-ids [complex type]

An array of Network Security Group OCIDs used to define network access for either Deployments or Connections. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--rest-auth-secret-key [text]

The BASE64URL-encoded HMAC secret key used to sign REST submission requests when the Spark master’s REST endpoint is protected by the JWS filter. Applicable only when submitMode is REST and the cluster enforces REST request signing. This setting is independent of authenticationType.

Deprecated: This field is deprecated and replaced by “restAuthSecretKeySecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--rest-auth-secret-key-secret-id [text]

The OCID of the Secret that stores the REST submission signing key. Applicable only when submitMode is REST and the cluster enforces REST request signing. This setting is independent of authenticationType. Note: When provided, ‘restAuthSecretKey’ field must not be provided.

--rest-endpoint [text]

Spark standalone REST submission base URL. e.g.: ‘https://spark-master.example.com:6066’ Applicable only and required when submitMode is REST.

--routing-method [text]

Controls the network traffic direction to the target: SHARED_DEPLOYMENT_ENDPOINT: Network traffic flows from the assigned deployment’s private endpoint through the deployment’s subnet. DEDICATED_ENDPOINT: A dedicated private endpoint is created in the target VCN subnet for the connection. The subnetId is required when DEDICATED_ENDPOINT networking is selected. SHARED_SERVICE_ENDPOINT: Traffic flows through the Goldengate Service’s network to public hosts. Cannot be used for private targets.

Deprecated: SHARED_SERVICE_ENDPOINT is deprecated. Use another supported routingMethod value, or update existing connections to use a supported routing method. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

Accepted values are:

DEDICATED_ENDPOINT, SHARED_DEPLOYMENT_ENDPOINT, SHARED_SERVICE_ENDPOINT
--rpc-cipher [text]

Cipher transform used for Spark RPC encryption. Applicable when securityProtocol is AES_RPC. Defaults to AES_CTR_NOPADDING when not provided.

--security-attributes [complex type]

Security attributes for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags.

Example:

{"Oracle-ZPR": {"MaxEgressCount": {"value": "42", "mode": "enforce"}}}

This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.

The --generate-param-json-input option can be used to generate an example of the JSON which must be provided. We recommend storing this example in a file, modifying it as needed and then passing it back in via the file:// syntax.

--ssl-key-password [text]

Password for the certificate private key inside the client key store. Provide this only when it differs from the key store password. Applicable when securityProtocol is TLS and a client key store is provided.

Deprecated: This field is deprecated and replaced by “sslKeyPasswordSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--ssl-key-password-secret-id [text]

The OCID of the Secret that stores the private key password. Applicable when securityProtocol is TLS and a client key store is provided. Note: When provided, ‘sslKeyPassword’ field must not be provided.

--subnet-id [text]

The OCID of the target subnet of the dedicated connection.

--subscription-id [text]

The OCID of the subscription with which resource needs to be associated with.

--tls-protocol [text]

TLS protocol negotiated by the Spark client. Applicable when securityProtocol is TLS. Defaults to TLS_V1_3 when not provided. Must be supported by the target cluster.

--trust-store [text]

The base64 encoded content of the trust store used to validate the Spark server certificate. Applicable when securityProtocol is TLS. Provide this when the server certificate is not already trusted by the JVM.

Deprecated: This field is deprecated and replaced by “trustStoreSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--trust-store-password [text]

Password for the trust store. Applicable when securityProtocol is TLS and a trust store is provided. Required when ‘trustStore’ is provided.

Deprecated: This field is deprecated and replaced by “trustStorePasswordSecretId”. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

--trust-store-password-secret-id [text]

The OCID of the Secret that stores the trust store password. Applicable when securityProtocol is TLS and a trust store is provided. Required when ‘trustStoreSecretId’ is provided. Note: When provided, ‘trustStorePassword’ field must not be provided.

--trust-store-secret-id [text]

The OCID of the Secret that stores the trust store content. Applicable when securityProtocol is TLS. Provide this when the server certificate is not already trusted by the JVM. Note: When provided, ‘trustStore’ field must not be provided.

--trust-store-type [text]

Trust store format used by the Spark client. Applicable when securityProtocol is TLS and a trust store is provided. Defaults to JKS when not provided. Provide this only when a trust store is provided.

--vault-id [text]

References the OCI Vault that contains the customer-managed encryption key identified by keyId.

Deprecated: This field is deprecated for GoldenGate connections. Sensitive attributes should be provided using the corresponding Secret OCID attributes of the connection (for example, passwordSecretId) instead of plain-text attributes encrypted with vaultId and keyId. This change follows the GoldenGate “Plain Text Fields in Connections” deprecation: https://docs.oracle.com/en-us/iaas/Content/servicechanges.htm#servicechanges_topic-GoldenGate

This field is applicable only when doesUseSecretIds is set to false. If vaultId is provided, keyId must also be provided.

--wait-for-state [text]

This operation asynchronously creates, modifies or deletes a resource and uses a work request to track the progress of the operation. Specify this option to perform the action and then wait until the work request reaches a certain state. Multiple states can be specified, returning on the first state. For example, --wait-for-state ACCEPTED --wait-for-state CANCELED would return on whichever lifecycle state is reached first. If timeout is reached, a return code of 2 is returned. For any other error, a return code of 1 is returned.

Accepted values are:

ACCEPTED, CANCELED, FAILED, IN_PROGRESS, SUCCEEDED
--wait-interval-seconds [integer]

Check every --wait-interval-seconds to see whether the work request has reached the state defined by --wait-for-state. Defaults to 30 seconds.

Example using required parameter

Copy and paste the following example into a JSON file, replacing the example parameters with your own.

    oci goldengate connection create-spark-connection --generate-param-json-input artifact-location > artifact-location.json

Copy the following CLI commands into a file named example.sh. Run the command by typing “bash example.sh” and replacing the example parameters with your own.

Please note this sample will only work in the POSIX-compliant bash-like shell. You need to set up the OCI configuration and appropriate security policies before trying the examples.

    export authentication_type=<substitute-value-of-authentication_type> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-authentication-type
    export cluster_endpoint=<substitute-value-of-cluster_endpoint> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-cluster-endpoint
    export compartment_id=<substitute-value-of-compartment_id> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-compartment-id
    export display_name=<substitute-value-of-display_name> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-display-name
    export security_protocol=<substitute-value-of-security_protocol> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-security-protocol
    export submit_mode=<substitute-value-of-submit_mode> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-submit-mode
    export technology_type=<substitute-value-of-technology_type> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/goldengate/connection/create-spark-connection.html#cmdoption-technology-type

    oci goldengate connection create-spark-connection --artifact-location file://artifact-location.json --authentication-type $authentication_type --cluster-endpoint $cluster_endpoint --compartment-id $compartment_id --display-name $display_name --security-protocol $security_protocol --submit-mode $submit_mode --technology-type $technology_type