Running CalculiX on Amazon EC2 cloud service

Many engineering problems require powerful PC to run FEA. It is not true for regular CAD work (for example in Solidworks) - having average (not very expensive computer) and general computer knowledge you still able to work with comfort (if you do not like to run many programs at once, because actually many browsers and messengers are eating more memory than powerful CAD).
But FEA software require usually require powerful machines to be able to solve complex nonlinear problems with contact, pretensioned bolts and etc.
At home I am using relatively old laptop with 4 cores and 8 GB of memory under Ubuntu 14 and solving of some problems is not so easy, if they can't be simplified. In this case there is relatively cheap solution, that is only need some money + Internet connection (not necessarily very fast).
You may rent remote computer on Amazon, Google (not sure, but probably Microsoft) and enjoy the power for relatively cheap cost.
Here is the description about running Caelinux, using cloud service (for European users). But I want to describe simple way to run just CalculiX on regular Ubuntu remotely.  
At first get registration at here https://aws.amazon.com/ 
setup your login, password and payment options.
Then go to https://aws.amazon.com/ec2/ 
and get started. I don't want to describe it in all details, read the information on website. But I can tell just two things. You should create and save security .pem file and select VM instance for your first test. At the beginning it can be weak and slow machine with minimum of memory, that is available for free (for some amount of hours). It is more than enough for test. Also select Ubuntu for OS.
My description below is about running remote Ubuntu from my home Ubuntu but basically you can do it from Windows or any other OS (using PuTTY or web-browser). 

1) At first, you have to connect to remote Ubuntu. I am doing it from terminal emulator:

sudo ssh -i /path/to/saved/file.pem ubuntu@ecX-XX-XX-XX-XXX.us-east-2.compute.amazonaws.com

where "ecX-XX-XX-XX-XXX.us-east-2.compute.amazonaws.com" can be copy-pased from EC2 management web-page  - Public DNS (IPv4).















Enter your local ubuntu password (after "sudo") and type yes, when it is asking "Are you sure you want to continue connecting?". Now your terminal emulator is serving for remote VM Ubuntu linux. Saying first, it is completely pure linux, just as command line and it doesn't have even one user (you're root). Here what you can do - is to create user, and then install Ubuntu GUI and special remote desktop software like x2go or No-Machine. Make search about that, like: "amazon GUI linux"
This feature (remote linux) is usually for professional computer specialists, servers, web-applications and things like that, and they need nothing except command line. Using remote machine for FEA and numerical calculation is not so typical. 
But we won't use GUI because it is actually not needed. All you need is to get some practice for 2-3 linux commands, described below.

 2) Let's install CCX (solver) on remote VM. Statically linked version, provided with CalculiX launcer doesn't require installation. We will just copy it with card like: 

wget "https://drive.google.com/uc?export=download&id=0B1dVWvbd4cYgOFBjUS1vSzN6M0k" -O ccx

wget means download from web, and in this case I am using modified link for my GoogleDrive (you can also use it, or share ccx executable from /bin  folder on your own drive. The sharing link should be modified with prefix:

https://drive.google.com/uc?export=download&id=

Otherwise you will download html page from google. Also this trouble may happen with large files, so learn how to use .tar.gz archive (typical type for linux) and use wget for archive.
If you have access to your own ftp - use it instead of google drive. You're free to search and read about "wget" in the internet (this is just linux shell command)

3) Get the test sample, for example:

wget "https://drive.google.com/uc?export=download&id=0B1dVWvbd4cYgXzBkcS1MdU9JZ2s" -O test.inp

This is the testing text file test.inp
Allow executing of CCX

chmod +x ccx

and run test.

./ccx test

You should be able to see typical ccx output and "Job is Finished"

4)  the whole reason why we use cloud service - is powerful computer with many cores. You should create environment variable for ubuntu, as it is described in the ReadMe(ubuntu).pdf in CalculiX Launcher. This is also described in CCX help files. Type:

touch ~/.pam_environment

and then

nano ~/.pam_environment

(nano is the text editor for Ubuntu)
Remember, that you're root on VM and "sudo" is not needed. 
Create two lines in the text file: 

OMP_NUM_THREADS=64
export OMP_NUM_THREADS


64 is maximal number of cores, available on remote PC (for money).
Then do:
ctrl+O
ctrl+X

(save and exit from nano)
After restart of remote VM you should be able to see how many cores you have in calculation (CCX output).
5) For real problems the inp file can be uploaded on google drive as tar.gz arcive, you may get it from there, for example:


wget "https://drive.google.com/uc?export=download&confirm&id=BUNCHOFNUMBERSANDLETTERS" -O static5.tar.gz
Untar it on VM:

tar -zxvf static5.tar.gz

After calculation is done and it was successful  you can make archive from frd (results) file before you will pull it back on your home machine for post-processing.

tar -zcvf static5.tar.gz static5.frd  


6) Final step - is getting archive file with results from VM to home. There are bunch of ways to do it, but I am using command like:

sudo scp -i /path/to/saved/file.pem ubuntu@ecX-XX-XX-XX-XXX.us-east-2.compute.amazonaws.com:static5.tar.gz /home/me/Desktop 

This card is running in separate terminal emulator on my local machine, but also it can be used on VM instance. Read more about secure copy, linux shell, for example here: http://www.hypexr.org/linux_scp_help.php

7) Few general notes:
- you can prepare all linux shell commands in separate text file, to copy-paste them then without typing.
- ecX-XX-XX-XX-XXX.us-east-2.compute.amazonaws.com is variable for each session of VM instance. Get it from amazon web-page.
-some additional linux shell commands, like "file", "nano" can be useful for blind work on remote VM. This information can be found everywhere in web. It is not something special about amazon or CalculiX.





1 comment: