Data Science in Cloud

Introduction

There are times when we are restricted by the computing power of the local machine to train the machine learning model. So as an alternative we can train our model in the cloud where we can get unlimited computing power.

In this post, we will use AWS (amazon web services) as the cloud platform to train the machine learning model in the cloud. We will start by running up an EC2 instance, creating a Deep learning AMI, and finally running jupyter from local thereby running code on local but leveraging the unlimited computing power of the cloud.

Follow the below steps to set up an EC2 instance and connect from the local jupyter to train a machine learning model in the cloud 

• Start an EC2 instance by logging in the AWS platform and choose any Amazon machine image (AMI)

• Select a GPU instance based on the data to be processed and click on configure instance details.

• Click on Launch and select your existing Keypair.

• EC2 instance will start running.

• To connect from Local through a Jupyter notebook, open cmd prompt to ssh GPU instance. Please make sure the keypair file(filename.pem) should exist in the same folder, otherwise, pass the keyPair file with the complete path.

ssh -i <key-pair_filename>.pem ubuntu@<IP>

1. Run below command in command prompt to start Jupyter on the GPU instance. and save the token

jupyter notebook --no-browser --port=8889

• Open a new command prompt to redirect Jupyter to the running GPU instance from the Local.

ssh -i <key-pair_filename>.pem -L 8002:localhost:8889 ubuntu@<ip>

• Open a web browser and go to http://localhost:8002/tree/ and use the token that we saved earlier.

• Jupyter notebook should start running, click on New to choose the environment.

• We can start using Jupyter notebook backed by GPU instance for analysis.

• Don’t forget to turn off instance after use, go back to AWS, click on Actions → Instance State → Terminate.

Conclusion

In this post, we explored how to set up an environment for training machine learning models in the cloud.