# API Reference
data.fluid.io/v1alpha1
Package v1alpha1 is the v1alpha1 version of the API.
Resource Types:AlluxioRuntime
AlluxioRuntime is the Schema for the alluxioruntimes API
| Field | Description | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
apiVersionstring |
data.fluid.io/v1alpha1
| ||||||||||||||||||||||||||||||||
kindstring | AlluxioRuntime | ||||||||||||||||||||||||||||||||
metadataKubernetes meta/v1.ObjectMeta |
Refer to the Kubernetes API documentation for the fields of the
metadata field.
| ||||||||||||||||||||||||||||||||
specAlluxioRuntimeSpec |
| ||||||||||||||||||||||||||||||||
statusRuntimeStatus |
DataBackup
DataBackup is the Schema for the backup API
| Field | Description | ||||||
|---|---|---|---|---|---|---|---|
apiVersionstring |
data.fluid.io/v1alpha1
| ||||||
kindstring | DataBackup | ||||||
metadataKubernetes meta/v1.ObjectMeta |
Refer to the Kubernetes API documentation for the fields of the
metadata field.
| ||||||
specDataBackupSpec |
| ||||||
statusDataBackupStatus |
DataLoad
DataLoad is the Schema for the dataloads API
| Field | Description | ||||||
|---|---|---|---|---|---|---|---|
apiVersionstring |
data.fluid.io/v1alpha1
| ||||||
kindstring | DataLoad | ||||||
metadataKubernetes meta/v1.ObjectMeta |
Refer to the Kubernetes API documentation for the fields of the
metadata field.
| ||||||
specDataLoadSpec |
| ||||||
statusDataLoadStatus |
Dataset
Dataset is the Schema for the datasets API
| Field | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
apiVersionstring |
data.fluid.io/v1alpha1
| ||||||||||||||||
kindstring | Dataset | ||||||||||||||||
metadataKubernetes meta/v1.ObjectMeta |
Refer to the Kubernetes API documentation for the fields of the
metadata field.
| ||||||||||||||||
specDatasetSpec |
| ||||||||||||||||
statusDatasetStatus |
JindoRuntime
JindoRuntime is the Schema for the jindoruntimes API
| Field | Description | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
apiVersionstring |
data.fluid.io/v1alpha1
| ||||||||||||||||||||||
kindstring | JindoRuntime | ||||||||||||||||||||||
metadataKubernetes meta/v1.ObjectMeta |
Refer to the Kubernetes API documentation for the fields of the
metadata field.
| ||||||||||||||||||||||
specJindoRuntimeSpec |
| ||||||||||||||||||||||
statusRuntimeStatus |
APIGatewayStatus
(Appears on: RuntimeStatus)
API Gateway
| Field | Description |
|---|---|
endpointstring | Endpoint for accessing |
AlluxioCompTemplateSpec
(Appears on: AlluxioRuntimeSpec)
AlluxioCompTemplateSpec is a description of the Alluxio commponents
| Field | Description |
|---|---|
replicasint32 | (Optional) Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster |
jvmOptions[]string | Options for JVM |
propertiesmap[string]string | (Optional) Configurable properties for the Alluxio component. |
portsmap[string]int | (Optional) Ports used by Alluxio(e.g. rpc: 19998 for master) |
resourcesKubernetes core/v1.ResourceRequirements | (Optional) Resources that will be requested by the Alluxio component. |
envmap[string]string | Environment variables that will be used by Alluxio component. |
enabledbool | (Optional) Enabled or Disabled for the components. For now, only API Gateway is enabled or disabled. |
AlluxioFuseSpec
(Appears on: AlluxioRuntimeSpec)
AlluxioFuseSpec is a description of the Alluxio Fuse
| Field | Description |
|---|---|
imagestring | Image for Alluxio Fuse(e.g. alluxio/alluxio-fuse) |
imageTagstring | Image Tag for Alluxio Fuse(e.g. 2.3.0-SNAPSHOT) |
imagePullPolicystring | One of the three policies: |
jvmOptions[]string | Options for JVM |
propertiesmap[string]string | Configurable properties for Alluxio System. |
envmap[string]string | Environment variables that will be used by Alluxio Fuse |
resourcesKubernetes core/v1.ResourceRequirements | (Optional) Resources that will be requested by Alluxio Fuse. |
args[]string | Arguments that will be passed to Alluxio Fuse |
globalbool | (Optional) If the fuse client should be deployed in global mode, otherwise the affinity should be considered |
nodeSelectormap[string]string | (Optional) NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled |
AlluxioRuntimeRole
(string alias)
AlluxioRuntimeSpec
(Appears on: AlluxioRuntime)
AlluxioRuntimeSpec defines the desired state of AlluxioRuntime
| Field | Description |
|---|---|
alluxioVersionVersionSpec | The version information that instructs fluid to orchestrate a particular version of Alluxio. |
masterAlluxioCompTemplateSpec | Desired state for Alluxio master |
jobMasterAlluxioCompTemplateSpec | Desired state for Alluxio job master |
workerAlluxioCompTemplateSpec | Desired state for Alluxio worker |
jobWorkerAlluxioCompTemplateSpec | Desired state for Alluxio job Worker |
apiGatewayAlluxioCompTemplateSpec | Desired state for Alluxio API Gateway |
initUsersInitUsersSpec | The spec of init users |
fuseAlluxioFuseSpec | Desired state for Alluxio Fuse |
propertiesmap[string]string | Configurable properties for Alluxio system. |
jvmOptions[]string | Options for JVM |
tieredstoreTieredstore | Tiered storage used by Alluxio |
dataData | Management strategies for the dataset to which the runtime is bound |
replicasint32 | The replicas of the worker, need to be specified |
runAsUser | Manage the user to run Alluxio Runtime |
disablePrometheusbool | (Optional) Disable monitoring for Alluxio Runtime Promethous is enabled by default |
hadoopConfigstring | (Optional) Name of the configMap used to support HDFS configurations when using HDFS as Alluxio’s UFS. The configMap must be in the same namespace with the AlluxioRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value. |
BackupLocation
(Appears on: DataBackupStatus)
BackupLocation describes the final backup location of DataBackup
| Field | Description |
|---|---|
pathstring | Path describes the path of backup, in the form of local:///absolutePath or pvc://pvcName/subpath |
nodeNamestring | NodeName describes the nodeName of backup if Path is in the form of local://subpath |
CacheableNodeAffinity
(Appears on: DatasetSpec)
CacheableNodeAffinity defines constraints that limit what nodes this dataset can be cached to.
| Field | Description |
|---|---|
requiredKubernetes core/v1.NodeSelector | Required specifies hard node constraints that must be met. |
Condition
(Appears on: DataBackupStatus, DataLoadStatus)
Condition explains the transitions on phase
| Field | Description |
|---|---|
typecommon.ConditionType | Type of condition, either |
statusKubernetes core/v1.ConditionStatus | Status of the condition, one of |
reasonstring | Reason for the condition’s last transition |
messagestring | Message is a human-readable message indicating details about the transition |
lastProbeTimeKubernetes meta/v1.Time | LastProbeTime describes last time this condition was updated. |
lastTransitionTimeKubernetes meta/v1.Time | LastTransitionTime describes last time the condition transitioned from one status to another. |
Data
(Appears on: AlluxioRuntimeSpec)
Data management strategies
| Field | Description |
|---|---|
replicasint32 | (Optional) The copies of the dataset |
pinbool | (Optional) Pin the dataset or not. Refer to Alluxio User-CLI pin |
DataBackupSpec
(Appears on: DataBackup)
DataBackupSpec defines the desired state of DataBackup
| Field | Description |
|---|---|
datasetstring | Dataset defines the target dataset of the DataBackup |
backupPathstring | BackupPath defines the target path to save data of the DataBackup |
runAsUser | Manage the user to run Alluxio DataBackup |
DataBackupStatus
(Appears on: DataBackup)
DataBackupStatus defines the observed state of DataBackup
| Field | Description |
|---|---|
phasecommon.Phase | Phase describes current phase of DataBackup |
backupLocationBackupLocation | BackupLocation tell user the location to save data of the DataBackup |
durationstring | Duration tell user how much time was spent to backup |
conditions[]Condition | Conditions consists of transition information on DataBackup’s Phase |
DataLoadSpec
(Appears on: DataLoad)
DataLoadSpec defines the desired state of DataLoad
| Field | Description |
|---|---|
datasetTargetDataset | Dataset defines the target dataset of the DataLoad |
loadMetadatabool | LoadMetadata specifies if the dataload job should load metadata |
target[]TargetPath | Target defines target paths that needs to be loaded |
DataLoadStatus
(Appears on: DataLoad)
DataLoadStatus defines the observed state of DataLoad
| Field | Description |
|---|---|
phasecommon.Phase | Phase describes current phase of DataLoad |
conditions[]Condition | Conditions consists of transition information on DataLoad’s Phase |
durationstring | Duration tell user how much time was spent to load the data |
DataRestoreLocation
(Appears on: DatasetSpec)
DataRestoreLocation describes the spec restore location of Dataset
| Field | Description |
|---|---|
pathstring | (Optional) Path describes the path of restore, in the form of local://subpath or pvc://pvcName/subpath |
nodeNamestring | (Optional) NodeName describes the nodeName of restore if Path is in the form of local://subpath |
DatasetCondition
(Appears on: DatasetStatus)
Condition describes the state of the cache at a certain point.
| Field | Description |
|---|---|
typeDatasetConditionType | Type of cache condition. |
statusKubernetes core/v1.ConditionStatus | Status of the condition, one of True, False, Unknown. |
reasonstring | The reason for the condition’s last transition. |
messagestring | A human readable message indicating details about the transition. |
lastUpdateTimeKubernetes meta/v1.Time | The last time this condition was updated. |
lastTransitionTimeKubernetes meta/v1.Time | Last time the condition transitioned from one status to another. |
DatasetSpec
(Appears on: Dataset)
DatasetSpec defines the desired state of Dataset
| Field | Description |
|---|---|
mounts[]Mount | Mount Points to be mounted on Alluxio. |
ownerUser | (Optional) The owner of the dataset |
nodeAffinityCacheableNodeAffinity | (Optional) NodeAffinity defines constraints that limit what nodes this dataset can be cached to. This field influences the scheduling of pods that use the cached dataset. |
tolerations[]Kubernetes core/v1.Toleration | (Optional) If specified, the pod’s tolerations. |
accessModes[]Kubernetes core/v1.PersistentVolumeAccessMode | (Optional) AccessModes contains all ways the volume backing the PVC can be mounted |
runtimes[]Runtime | Runtimes for supporting dataset (e.g. AlluxioRuntime) |
placementPlacementMode | (Optional) Manage switch for opening Multiple datasets single node deployment or not TODO(xieydd) In future, evaluate node resources and runtime resources to decide whether to turn them on |
dataRestoreLocationDataRestoreLocation | (Optional) DataRestoreLocation is the location to load data of dataset been backuped |
DatasetStatus
(Appears on: Dataset)
DatasetStatus defines the observed state of Dataset
| Field | Description |
|---|---|
ufsTotalstring | Total in GB of dataset in the cluster |
phaseDatasetPhase | Dataset Phase. One of the four phases: |
runtimes[]Runtime | Runtimes for supporting dataset |
conditions[]DatasetCondition | Conditions is an array of current observed conditions. |
cacheStatescommon.CacheStateList | CacheStatus represents the total resources of the dataset. |
hcfsHCFSStatus | HCFSStatus represents hcfs info |
fileNumstring | FileNum represents the file numbers of the dataset |
dataLoadRefstring | DataLoadRef specifies the running DataLoad job that targets this Dataset. This is mainly used as a lock to prevent concurrent DataLoad jobs. |
dataBackupRefstring | DataBackupRef specifies the running Backup job that targets this Dataset. This is mainly used as a lock to prevent concurrent DataBackup jobs. |
EncryptOption
(Appears on: Mount)
| Field | Description |
|---|---|
namestring | The name of encryptOption |
valueFromEncryptOptionSource | (Optional) The valueFrom of encryptOption |
EncryptOptionSource
(Appears on: EncryptOption)
| Field | Description |
|---|---|
secretKeyRefSecretKeySelector | (Optional) The encryptInfo obtained from secret |
HCFSStatus
(Appears on: DatasetStatus)
HCFS Endpoint info
| Field | Description |
|---|---|
endpointstring | Endpoint for accessing |
underlayerFileSystemVersionstring | Underlayer HCFS Compatible Version |
InitUsersSpec
(Appears on: AlluxioRuntimeSpec)
InitUsersSpec is a description of the initialize the users for runtime
| Field | Description |
|---|---|
imagestring | Image for initialize the users for runtime(e.g. alluxio/alluxio-User init) |
imageTagstring | Image Tag for initialize the users for runtime(e.g. 2.3.0-SNAPSHOT) |
imagePullPolicystring | One of the three policies: |
envmap[string]string | Environment variables that will be used by initialize the users for runtime |
resourcesKubernetes core/v1.ResourceRequirements | (Optional) Resources that will be requested by initialize the users for runtime. |
JindoCompTemplateSpec
(Appears on: JindoRuntimeSpec)
JindoCompTemplateSpec is a description of the Jindo commponents
| Field | Description |
|---|---|
replicasint32 | (Optional) Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster |
propertiesmap[string]string | (Optional) Configurable properties for the Jindo component. |
portsmap[string]int | (Optional) |
resourcesKubernetes core/v1.ResourceRequirements | (Optional) Resources that will be requested by the Jindo component. |
envmap[string]string | Environment variables that will be used by Jindo component. |
nodeSelectormap[string]string | (Optional) NodeSelector is a selector which must be true for the master to fit on a node |
JindoFuseSpec
(Appears on: JindoRuntimeSpec)
JindoFuseSpec is a description of the Jindo Fuse
| Field | Description |
|---|---|
imagestring | Image for Jindo Fuse(e.g. jindo/jindo-fuse) |
imageTagstring | Image Tag for Jindo Fuse(e.g. 2.3.0-SNAPSHOT) |
imagePullPolicystring | One of the three policies: |
propertiesmap[string]string | Configurable properties for Jindo System. |
envmap[string]string | Environment variables that will be used by Jindo Fuse |
resourcesKubernetes core/v1.ResourceRequirements | (Optional) Resources that will be requested by Jindo Fuse. |
args[]string | Arguments that will be passed to Jindo Fuse |
globalbool | (Optional) If the fuse client should be deployed in global mode, otherwise the affinity should be considered |
nodeSelectormap[string]string | (Optional) NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled |
JindoRuntimeSpec
(Appears on: JindoRuntime)
JindoRuntimeSpec defines the desired state of JindoRuntime
| Field | Description |
|---|---|
jindoVersionVersionSpec | The version information that instructs fluid to orchestrate a particular version of Jindo. |
masterJindoCompTemplateSpec | Desired state for Jindo master |
workerJindoCompTemplateSpec | Desired state for Jindo worker |
fuseJindoFuseSpec | Desired state for Jindo Fuse |
propertiesmap[string]string | Configurable properties for Jindo system. |
tieredstoreTieredstore | Tiered storage used by Jindo |
replicasint32 | The replicas of the worker, need to be specified |
runAsUser | Manage the user to run Jindo Runtime |
userstring | |
hadoopConfigstring | (Optional) Name of the configMap used to support HDFS configurations when using HDFS as Jindo’s UFS. The configMap must be in the same namespace with the JindoRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value. |
secretstring |
Level
(Appears on: Tieredstore)
Level describes configurations a tier needs.
Refer to Configuring Tiered Storage for more info
| Field | Description |
|---|---|
mediumtypecommon.MediumType | Medium Type of the tier. One of the three types: |
pathstring | File paths to be used for the tier. Multiple paths are supported. Multiple paths should be separated with comma. For example: “/mnt/cache1,/mnt/cache2”. |
quotaKubernetes resource.Quantity | (Optional) Quota for the whole tier. (e.g. 100Gi) Please note that if there’re multiple paths used for this tierstore, the quota will be equally divided into these paths. If you’d like to set quota for each, path, see QuotaList for more information. |
quotaListstring | (Optional) QuotaList are quotas used to set quota on multiple paths. Quotas should be separated with comma. Quotas in this list will be set to paths with the same order in Path. For example, with Path defined with “/mnt/cache1,/mnt/cache2” and QuotaList set to “100Gi, 50Gi”, then we get 100GiB cache storage under “/mnt/cache1” and 50GiB under “/mnt/cache2”. Also note that num of quotas must be consistent with the num of paths defined in Path. |
highstring | Ratio of high watermark of the tier (e.g. 0.9) |
lowstring | Ratio of low watermark of the tier (e.g. 0.7) |
Mount
(Appears on: DatasetSpec)
Mount describes a mounting.
Refer to Alluxio Storage Integrations for more info
| Field | Description |
|---|---|
mountPointstring | MountPoint is the mount point of source. |
optionsmap[string]string | (Optional) The Mount Options. |
namestring | The name of mount |
pathstring | (Optional) The path of mount, if not set will be /{Name} |
readOnlybool | (Optional) Optional: Defaults to false (read-write). |
sharedbool | (Optional) Optional: Defaults to false (shared). |
encryptOptions[]EncryptOption | (Optional) The secret information |
PlacementMode
(string alias)
(Appears on: DatasetSpec)
Runtime
(Appears on: DatasetSpec, DatasetStatus)
Runtime describes a runtime to be used to support dataset
| Field | Description |
|---|---|
namestring | Name of the runtime object |
namespacestring | Namespace of the runtime object |
categorycommon.Category | Category the runtime object belongs to (e.g. Accelerate) |
typestring | Runtime object’s type (e.g. Alluxio) |
RuntimeCondition
(Appears on: RuntimeStatus)
Condition describes the state of the cache at a certain point.
| Field | Description |
|---|---|
typeRuntimeConditionType | Type of cache condition. |
statusKubernetes core/v1.ConditionStatus | Status of the condition, one of True, False, Unknown. |
reasonstring | The reason for the condition’s last transition. |
messagestring | A human readable message indicating details about the transition. |
lastProbeTimeKubernetes meta/v1.Time | The last time this condition was updated. |
lastTransitionTimeKubernetes meta/v1.Time | Last time the condition transitioned from one status to another. |
RuntimeStatus
(Appears on: AlluxioRuntime, JindoRuntime)
RuntimeStatus defines the observed state of Runtime
| Field | Description |
|---|---|
valueFilestring | config map used to set configurations |
masterPhaseRuntimePhase | MasterPhase is the master running phase |
masterReasonstring | Reason for Master’s condition transition |
workerPhaseRuntimePhase | WorkerPhase is the worker running phase |
workerReasonstring | Reason for Worker’s condition transition |
desiredWorkerNumberScheduledint32 | The total number of nodes that should be running the runtime worker pod (including nodes correctly running the runtime worker pod). |
currentWorkerNumberScheduledint32 | The total number of nodes that can be running the runtime worker pod (including nodes correctly running the runtime worker pod). |
workerNumberReadyint32 | The number of nodes that should be running the runtime worker pod and have one or more of the runtime worker pod running and ready. |
workerNumberAvailableint32 | (Optional) The number of nodes that should be running the runtime worker pod and have one or more of the runtime worker pod running and available (ready for at least spec.minReadySeconds) |
workerNumberUnavailableint32 | (Optional) The number of nodes that should be running the runtime worker pod and have none of the runtime worker pod running and available (ready for at least spec.minReadySeconds) |
desiredMasterNumberScheduledint32 | The total number of nodes that should be running the runtime pod (including nodes correctly running the runtime master pod). |
currentMasterNumberScheduledint32 | The total number of nodes that should be running the runtime pod (including nodes correctly running the runtime master pod). |
masterNumberReadyint32 | The number of nodes that should be running the runtime worker pod and have zero or more of the runtime master pod running and ready. |
fusePhaseRuntimePhase | FusePhase is the Fuse running phase |
fuseReasonstring | Reason for the condition’s last transition. |
currentFuseNumberScheduledint32 | The total number of nodes that can be running the runtime Fuse pod (including nodes correctly running the runtime Fuse pod). |
desiredFuseNumberScheduledint32 | The total number of nodes that should be running the runtime Fuse pod (including nodes correctly running the runtime Fuse pod). |
fuseNumberReadyint32 | The number of nodes that should be running the runtime Fuse pod and have one or more of the runtime Fuse pod running and ready. |
fuseNumberUnavailableint32 | (Optional) The number of nodes that should be running the runtime fuse pod and have none of the runtime fuse pod running and available (ready for at least spec.minReadySeconds) |
fuseNumberAvailableint32 | (Optional) The number of nodes that should be running the runtime Fuse pod and have one or more of the runtime Fuse pod running and available (ready for at least spec.minReadySeconds) |
setupDurationstring | Duration tell user how much time was spent to setup the runtime |
conditions[]RuntimeCondition | Represents the latest available observations of a ddc runtime’s current state. |
cacheStatescommon.CacheStateList | CacheStatus represents the total resources of the dataset. |
selectorstring | Selector is used for auto-scaling |
apiGatewayAPIGatewayStatus | APIGatewayStatus represents rest api gateway status |
SecretKeySelector
(Appears on: EncryptOptionSource)
| Field | Description |
|---|---|
namestring | The name of required secret |
keystring | (Optional) The required key in the secret |
TargetDataset
(Appears on: DataLoadSpec)
TargetDataset defines the target dataset of the DataLoad
| Field | Description |
|---|---|
namestring | Name defines name of the target dataset |
namespacestring | Namespace defines namespace of the target dataset |
TargetPath
(Appears on: DataLoadSpec)
TargetPath defines the target path of the DataLoad
| Field | Description |
|---|---|
pathstring | Path defines path to be load |
replicasint32 | Replicas defines how many replicas will be loaded |
Tieredstore
(Appears on: AlluxioRuntimeSpec, JindoRuntimeSpec)
Tieredstore is a description of the tiered store
| Field | Description |
|---|---|
levels[]Level | configurations for multiple tiers |
User
(Appears on: AlluxioRuntimeSpec, DataBackupSpec, DatasetSpec, JindoRuntimeSpec)
User explains the user and group to run a Container
| Field | Description |
|---|---|
uidint64 | The uid to run the alluxio runtime |
gidint64 | The gid to run the alluxio runtime |
userstring | The user name to run the alluxio runtime |
groupstring | The group name to run the alluxio runtime |
VersionSpec
(Appears on: AlluxioRuntimeSpec, JindoRuntimeSpec)
VersionSpec represents the settings for the version that fluid is orchestrating.
| Field | Description |
|---|---|
imagestring | Image (e.g. alluxio/alluxio) |
imageTagstring | Image tag (e.g. 2.3.0-SNAPSHOT) |
imagePullPolicystring | One of the three policies: |
Generated with gen-crd-api-reference-docs
on git commit 01c0b49.
