Friday, October 14, 2016

Pivotal Cloud Foundry (PCF) Integration with Elastic Cloud Storage (ECS)

Recently, I was involved in integrating Pivotal Cloud Foundry with Elastic Cloud Storage (ECS), an object storage solution from EMC.

In this post, I'm going to document the hiccups we faced during this integration and how did we resolve this, so that it is easier for other folks who would like to carry out this integration.



References:

and the service broker code from git hub,https://github.com/spiegela/ecs-cf-service-broker



1. application.yml file Configuration:


First task would be to update the application.yml file in the broker code to have the correct configuration.

Note the spring profiles created in yml file. The active spring profile is defined in the build.gradle file to be 'development'. So based on that, we need to update the correct section in the yml file.

Under the broker section:

a) Provide a valid ECS namespace name (The namespace name is case sensitive). Under this namespace, PCF would create a bucket to store all the metadata related to this integration.


b) Provide a valid ECS replication group name (case sensitive).



c) Provide a management end point, which would generally be a https end point (with port 4443).
For eg, https://10.20.30.40:4443

d) Provide an object end point, which in our case was same as management end point (or /object/bucket suffix in the url)

e) Add a password property in this section and set it to the ECS password (This attribute is missing in the application.yml file, but you can see this property in the BrokerConfig.java file)

2. Enable SSL handshake communication:


Second task is to enable SSL handshake between the Service Broker and ECS. The broker uses the public key file localhost.pem file which is present in src/main/resources folder. We will need to replace this file with a public key file corresponding to our ECS installation.

Let's export the public certificate from our ECS application. 
a) Open the ECS application in a browser (say Chrome)

b) Follow these steps to save the certificate from ECS to local file system.
http://docs.bvstools.com/home/ssl-documentation/exporting-certificate-authorities-cas-from-a-website
c) Lets say in step b) above, the file was saved as ecscert.cer

d) Now , we will need to convert the public key file format from .cert to .pem. We will use Java keytool for this. There could be other tools for performing this step as well.

e) Run the following commands from command prompt. We are creating a sample keystore temporarily. We have named it as 'mytest', it would be temporary and doesnt matter. While creating this, it would ask for a password which should be remembered as it is required in further steps.
In the third command, below provide path to the ecscert.cer. If you are running these commands from same directory as the cert file, then provide the file name, otherwise complete path to the file.  
keytool -genkey -alias test -keystore mytest
keytool -delete -alias test -keystore mytest
keytool -import -trustcacerts -alias test -file ecscert.cer -keystore test.keystore
keytool -exportcert -alias test -file localhost.pem -rfc -keystore test.keystore
After the fourth command above, it would create a new file localhost.pem file which is what we would need.
f) Copy the above localhost.pem to src/main/resources and replace the existing localhost.pem file.

3. Service Broker security:


The service broker application uses spring security, so it uses a default username called 'user' and a password as defined in the below section (depending on the spring profile which you choose):

security:
  user:
    password: password
So, with the above config, the broker would be secured using credentials user/password.

4. Service Broker API Version:


Cloud Foundry comes with different Service Broker API versions and the broker application has to be compatible with it. This broker application uses API version 2.8, but your Cloud Foundry might expect a different version. You can use declare a bean to provide a new BrokerAPIVersion(). In our case, we simply set  the brokerApiVersion field property in BrokerConfig.java to 2.7. 


5. Push Service Broker app to Cloud Foundry:


The service broker application should be pushed to Cloud Foundry just like any other application. Sometimes is better to run this application locally to check if it is working fine.
a) Build the application using 'gradlew assemble'

b) Run the application using java -jar build/libs/ecs-cf-service-broker-0.0.1-SNAPSHOT.jar to see if it starts without any issues.

 c) Push the application to Cloud Foundry. We used memory of 750M for this.


6. Register Service Broker with Cloud Foundry:


Once the application is pushed successfully, we need to register the broker, so that it would appear in Cloud Foundry marketplace.
Run these commands after logging into CF CLI as admin.
a) cf create-service-broker ecs-broker user password https://ecs-broker-url
Note that the user & password above, are the broker credentials configured in step 3. The url is the service broker application URL, which we get after pushing to Cloud Foundry.

b) cf enable-service-access  ecs-namespace

c) cf enable-service-access  ecs-bucket 

d) cf marketplace
   
The fourth command above, 'cf marketplace' should display the ecs-broker service.


7. Verify Bucket Creation in ECS:


Login to ECS, go the namespace configured in Step 1. We would see a bucket 'ecs-cf-broker-repository'. This bucket was created by Cloud Foundry as part of integration.

Conclusion:

Bingo!, these steps would help us to successfully integrate ECS with Cloud Foundry and ready to rock and get ready to write cool Cloud Native applications using ECS Object Storage!!








12 comments:

  1. Thanks for letting us know very informative article. keep updating useful thoughts with us
    Web Hosting companies in India | VPS Hosting India | VPS Hosting Plans | VPS Hosting companies India

    ReplyDelete
  2. Thank you.. but article wrongly interpreted. as ECS is not cloud storage its Container Service !

    ReplyDelete
    Replies
    1. Hi, I think you got confused with Amazon ECS which is a container service. However, the ECS mentioned here is Elastic Cloud Storage solution from Dell EMC, which is an object storage solution like S3.

      Delete
  3. This blog is the general information for the feature. You got a good work for these blog.We have a developing our creative content of this mind.Thank you for this blog. This for very interesting and useful.

    rpa Training in Chennai

    rpa Training in bangalore

    rpa Training in pune

    blueprism Training in Chennai

    blueprism Training in bangalore

    blueprism Training in pune

    iot-training-in-chennai


    ReplyDelete
  4. Useful information.I am actual blessed to read this article.thanks for giving us this advantageous information.I acknowledge this post.and I would like bookmark this post.Thanks
    java training in marathahalli | java training in btm layout

    java training in jayanagar | java training in electronic city

    ReplyDelete
  5. Nice post. By reading your blog, i get inspired and this provides some useful information. Thank you for posting this exclusive post for our vision. 
    python training in annanagar
    python training in chennai
    python training in chennai
    python training in Bangalore

    ReplyDelete
  6. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
    Devops training in velachery
    Devops training in annanagar
    Devops training in tambaram
    DevOps online Training

    ReplyDelete
  7. I was looking for this certain information for a long time. Thank you and good luck.Blueprism online training

    Blue Prism Training in Pune

    ReplyDelete
  8. Does your blog have a contact page? I’m having problems locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing.
    AWS Training in Chennai |Best Amazon Web Services Training in Chennai
    AWS Training in Rajaji Nagar | Amazon Web Services Training in Rajaji Nagar

    Best AWS Training Institute in BTM Layout Bangalore ,AWS Coursesin BTM

    ReplyDelete

Quick Introduction to Terraform

Introduction There are many tools and frameworks available for provisioning infrastructure on the cloud. One of the most popular cloud a...