Skip to content

Commit ebb37f2

Browse files
committed
Updated readme and csd description
1 parent 9afe3a4 commit ebb37f2

File tree

3 files changed

+13
-12
lines changed

3 files changed

+13
-12
lines changed

.idea/vcs.xml

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,19 @@ This repository allows you to install [Apache Airflow](https://airflow.apache.or
55
## Requirements
66
- A supported operating system.
77
- MySQL or PostgreSQL database in which to store Airflow metadata.
8+
- RabbitMQ
89
- [Airflow](https://github.com/teamclairvoyant/apache-airflow-parcels)
910

1011
### Currently Supported Versions of Airflow
11-
- Airflow 1.7.1.3
12-
- Airflow 1.8.0
1312
- Airflow 1.9.0
13+
- Airflow 1.10.0
1414

1515
### Currently Supported Operating Systems
1616
- CentOS 6 & 7
1717
- RHEL 6 & 7
1818

1919
## Installing the CSD
20-
1. Download the Jar file. [Airflow CSD](http://archive.clairvoyantsoft.com/airflow/parcels/latest/AIRFLOW-{airflow_version}.jar)
20+
1. Download the Jar file. [Airflow 1.9.0 CSD](https://s3-us-west-2.amazonaws.com/archive.clairvoyantsoft.com/airflow/csd/AIRFLOW-1.9.0.jar)
2121
2. Copy the jar file to the `/opt/cloudera/csd` location on the Cloudera Manager server.
2222
3. Restart the Cloudera Manager Server service. `service cloudera-scm-server restart`
2323

@@ -141,10 +141,9 @@ jar -cvf AIRFLOW-1.0.0.jar -C src/ .
141141
```
142142

143143
## Limitations:
144-
1. Number of RabbitMQ instances is limited to 1.
145-
2. The IP address of the RabbitMQ instance has to be manually entered during installation configuration.
146-
3. After deploying configurations, there is no alert or warning that the specific roles needs to be restarted.
147-
4. Only 'airflow.contrib.auth.backends.password_auth' mechanism is supported for Airflow user authentication.
144+
1. The IP address of the RabbitMQ instance has to be manually entered during installation configuration.
145+
2. After deploying configurations, there is no alert or warning that the specific roles needs to be restarted.
146+
3. Only 'airflow.contrib.auth.backends.password_auth' mechanism is supported for Airflow user authentication.
148147

149148
## Future work:
150149
1. Build RabbitMQ parcel.
@@ -157,10 +156,6 @@ jar -cvf AIRFLOW-1.0.0.jar -C src/ .
157156

158157
Upon many deployments, you may face an error called 'Markup file already exists' while trying to stop a role and the process never stops. In that case, stop the process using the "Abort" command and navigate to `/var/run/cloudera-scm-agent/process` and delete all the `GracefulRoleStopRunner` directories.
159158

160-
### Lag in DAG Execution:
161-
162-
Occasionally, we experienced some delay in DAG execution. We are working to fix this.
163-
164159
## Resources:
165160
1. https://github.com/cloudera/cm_ext/wiki/The-Structure-of-a-CSD
166161
2. https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference

src/descriptor/service.sdl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
{
1515
"name" : "AIRFLOW",
1616
"label" : "Airflow",
17-
"description" : "Airflow is a platform to programmatically author, schedule and monitor workflows. <span class=\"error\">Before adding this service, ensure that the Airflow and RabbitMQ parcels have been activated.</span>",
17+
"description" : "Airflow is a platform to programmatically author, schedule and monitor workflows. <span class=\"error\">Before adding this service, please ensure that the Airflow parcel is activated and RabbitMQ is installed</span>",
1818
"version" : "1.9.0",
1919
"runAs" : {
2020
"user" : "root",

0 commit comments

Comments
 (0)