Wednesday, 1 May 2013

Stuck with Python 2.4


The goal
Connect to an Oracle AWS RDS database instance, using Python.



The challenge
Being stuck on a Linux platform with Python 2.4.3 and no root access (which is still quite common with many webhosts).



The solution
Below is step by step a description on how I solved this problem.


What Linux distribution are you running?
The commands below are nifty for finding out. You need to to determine what version of the Oracle Instant Client to download.

-bash-3.2$ cat /etc/*-release
CloudLinux Server release 5.9 (Sergey Oleynikov)
CloudLinux Server release 5.9 (Sergey Oleynikov)

-bash-3.2$ lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: CloudLinuxServer
Description: CloudLinux Server release 5.9 (Sergey Oleynikov)
Release: 5.9
Codename: SergeyOleynikov


Install Oracle Instant Client
Knowing the details about your OS (see above) you can download the correct version of Oracle Instant Client (which in the case above would be for Linux AMD64).

cd ~
mkdir .OIC

As you need to log into Oracle to download files using a simple wget command directly from the linux command line will not work. You can download the file manually, to another computer, and transfer it to your webhost (to the newly created ".OIC" folder) account using for example FileZilla, put the files into the folder you just created.

Download the following three packages:
- basic
- sqlplus
- sdk

After downloading your directory should look something like:

-bash-3.2$ ls -al
total 32152
drwxrwxr-x  3 klevstul klevstul     4096 Apr 28 03:57 .
drwx--x--x 35 klevstul klevstul     4096 Apr 28 12:55 ..
-rw-r--r--  1 klevstul klevstul 32200686 Apr 27 13:33 instantclient-basic-linuxAMD64-10.1.0.5.0-20060519.zip
-rw-r--r--  1 klevstul klevstul   294632 Apr 28 03:57 instantclient-sdk-linuxAMD64-10.1.0.5.0-20060519.zip
-rw-r--r--  1 klevstul klevstul   366963 Apr 27 13:37 instantclient-sqlplus-linuxAMD64-10.1.0.5.0-20060519.zip

Unzip all the files, using the "unzip" command, like:

-bash-3.2$ unzip instantclient-basic-linuxAMD64-10.1.0.5.0-20060519.zip

The unzipped files will end up in a directory, like "instantclient10_1".

Oracle Instant Client basic, sqlplus and sdk files.

Go to the instant client directory, and make some symlinks:

-bash-3.2$ cd $HOME/.OIC/instantclient10_1/
-bash-3.2$ ln -s libclntsh.so.10.1 libclntsh.so
-bash-3.2$ ln -s libocci.so.10.1 libocci.so


Environment variables
You need to add some environment variables. How to do this depends on the shell you're running. The below code is for the Bash shell (read more about setting, editing and deleting env var i Bash here).

-bash-3.2$ nano ~/.bashrc

Add following line to .bashrc to enable sqlplus and OIC working correctly:
export LD_LIBRARY_PATH=$HOME/.OIC/instantclient10_1

Load the new environment variables manually (to avoid having to log in and out):
-bash-3.2$ source ~/.bashrc


Test sqlplus
From Amazon's side you need to add the IP to your webhost in "DB Security Groups" to grant access to your database instance.


You might also have to contact your webhost support people, to have them opening up / enabling connecting to the port your database is running on (should be port 1521).

Having done this you can test sqlplus from the command line, to make sure you do have a working connection.

-bash-3.2$ ./sqlplus username/password@//privyinstance.cdhzvkd198ux.us-east-1.rds.amazonaws.com:1521/PRIVYDB

SQL*Plus: Release 10.1.0.5.0 - Production on Wed May 1 05:23:56 2013
Copyright (c) 1982, 2005, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Release 11.2.0.2.0 - 64bit Production
SQL>

Note that if you get the error message below it is due to not having set the LD_LIBRARY_PATH correctly:

-bash-3.2$ ./sqlplus sleeptracker/deepsleep@//privyinstance.cdhzvkd198ux.us-east-1.rds.amazonaws.com:1521/PRIVYDB
./sqlplus: error while loading shared libraries: libsqlplus.so: cannot open shared object file: No such file or directory


Install new version of Python
and the third party modules of your choice.
If you don't have a source directory (src) you can create one, or download somewhere else.

-bash-3.2$ cd ~/src/

-bash-3.2$ wget http://www.python.org/ftp/python/2.7.4/Python-2.7.4.tgz
-bash-3.2$ tar -zxvf Python-2.7.4.tgz
-bash-3.2$ cd Python-2.7.4
-bash-3.2$ mkdir ~/.localpython
-bash-3.2$ ./configure --prefix=/home/klevstul/.localpython
-bash-3.2$ make
-bash-3.2$ make install

-bash-3.2$ cd ~/src
-bash-3.2$ wget https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
-bash-3.2$ tar -zxvf virtualenv-1.9.1.tar.gz
-bash-3.2$ ~/.localpython/bin/python setup.py install

-bash-3.2$ mkdir ~/virtualenvs
-bash-3.2$ cd ~/virtualenvs
-bash-3.2$ ~/.localpython/bin/virtualenv py2.7 --python=/home/klevstul/.localpython/bin/python2.7

Add environment variable:
Add the location where the Python modules are downloaded to the environment (.bashrc - see above):
Add the following line to .bashrc and load it to the environement:
export PYTHONPATH=$HOME/.local/lib64/python2.4/site-packages/

Activate the environment:
-bash-3.2$ cd ~/virtualenvs/py2.7/bin/
-bash-3.2$ source ./activate
(py2.7)-bash-3.2$ python
Python 2.7.4 (default, Apr 28 2013, 12:39:28) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/site-packages/cx_Oracle-5.1.2-py2.7-linux-x86_64.egg', '/home/klevstul/.local/lib64/python2.4/site-packages', '/home/klevstul/virtualenvs/py2.7/lib/python27.zip', '/home/klevstul/virtualenvs/py2.7/lib/python2.7', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/plat-linux2', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/lib-tk', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/lib-old', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/lib-dynload', '/home/klevstul/.localpython/lib/python2.7', '/home/klevstul/.localpython/lib/python2.7/plat-linux2', '/home/klevstul/.localpython/lib/python2.7/lib-tk', '/home/klevstul/virtualenvs/py2.7/lib/python2.7/site-packages']
>>> import cx_Oracle
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named cx_Oracle
>>> exit()

Install needed modules:
(py2.7)-bash-3.2$ export LD_RUN_PATH=/home/klevstul/.OIC/instantclient10_1
(py2.7)-bash-3.2$ ~/virtualenvs/py2.7/bin/easy_install cx_Oracle
Searching for cx-Oracle
Best match: cx-Oracle 5.1.2
Processing cx_Oracle-5.1.2-py2.7-linux-x86_64.egg
Removing cx-Oracle 5.1.2 from easy-install.pth file
Adding cx-Oracle 5.1.2 to easy-install.pth file

Why setting LD_RUN_PATH?
Otherwise you'll end up getting this error (Internal Server Error 500) if you have no access to set the LD_LIBRARY_PATH variable in Apache.

Traceback (most recent call last):
  File "./jsonRpc.py", line 24, in <module>
    import cx_Oracle
  File "build/bdist.linux-x86_64/egg/cx_Oracle.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/cx_Oracle.py", line 6, in __bootstrap__
ImportError: libclntsh.so.10.1: cannot open shared object file: No such file or directory

ref:
http://stackoverflow.com/questions/5506110/it-is-possible-to-install-another-version-of-python-to-virtualenv


Update your Python script
The first line of your Python script should be modified so that the latest Python is used:

#! /home/klevstul/virtualenvs/py2.7/bin/python

Then you can connect to the Oracle DB, using cx_Oracle.



Alternative, using Perl
One alternative is to use Perl to connect to Oracle. As DBI was already installed I only had to setup "DBD::Oracle".

Add this to the environment (.bashrc):
eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`

Then load the file:
-bash-3.2$ source ~/.bashrc

Download the module and install it:
wget -O- http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
cpanm DBD::Oracle

ref:
http://stackoverflow.com/questions/2980297/how-can-i-use-cpan-as-a-non-root-user
http://stackoverflow.com/questions/12682783/regarding-perl-dbdoracle-installation-in-local-directory

No comments:

Post a Comment

Allowed HTML tags:
<b>bold</b>
<strong>strong</strong>
<i>italics</i>
<em>emphasis</em>
<a href="">hyperlink</a>


Please, show the courtesy of identifying yourself when adding a comment. Anonymous comments will, most likely, be removed.