Wednesday, July 29, 2015

How Python can talk to other language? aka Embedding & Extending; Call binaries within from Python


Lots of people have multiple times asked me below questions:
1. How to call binaries (or other programs) from a python script?
2. How to call python script from another program say a c code?
3. How to embed python in other language like C?
4. How to extend python with C language?

All above questions more or less fall in the same category, and roughly speaking trying to achieve same objective.

However, every time I have been asked above questions, first thing I did to identify the objective as to what particular is trying to achieve. Because most of the time, based on one's experience, job profile (like tester or coder) and programming language he is familiar with (if he is C or python programmer); one may ask how he can do X, but in reality his intention or objective is altogether a different.

So based on my experience so far, one can consider below things whenever he want python to talk to external program.


  1. At simplest, probably one might need to call a binary, within from the python. Or vice-a-versa python file from the c-program. Then one need not to bother about embedding or extending python.


Call a binary from python:
import os
cmd = 'bin/bar --option --otheroption'
os.system(cmd) # returns the exit status

OR if he wants fine grain control over child process then,

import subprocess
args = ("bin/bar", "-c", "somefile.xml", "-d", "text.txt", "-r", "aString", "-f", "anotherString")
#Or just:
#args = "bin/bar -c somefile.xml -d text.txt -r aString -f anotherString".split()
popen = subprocess.Popen(args, stdout=subprocess.PIPE)
popen.wait()
output = popen.stdout.read()
print output


Call a python program within from c program
system("python myfile.py")



  1. Now suppose, you already have python library and want to call any random function from that library. (in other words you want to embed python in C or other language)

Embedding Python in antoher language
#include "python.h"

int main(int argc, char *argv[])
{
  Py_SetProgramName(argv[0]);  /* optional but recommended */
  Py_Initialize();
  PyRun_SimpleString("from time import time,ctime\n"
                     "print 'Today is',ctime(time())\n");

  /*Or if you want to run python file within from the C code*/
  //pyRun_SimpleFile("Filename");
  Py_Finalize();
  return 0;
}


  1. Now lets talk about reverse case of above, that is you want to call C functions within from python program. If you want your C code, from then be part of your python it self, this can be called extending python's functionalities.  (Extending python using C is only possible in CPython version of the python)

Extending python can do two things that can’t be done directly in Python:
1. they can implement new built-in object types, and
2. they can call C library functions and system calls

Extending python can be done using Python APIs. However, this is not portable across all versions of the python, moreover its too complicated!. Hence it is advisable to use ctype, which is btw portable across various versions.

CType:
A windows example:

>> from ctypes import *
>> libc = cdll.msvcrt
>> print libc.time(None)
>> 1438069008
>> printf = libc.printf
>> printf("Hello, %s\n", "World!")
>> Hello, World!
14
>> printf("%d bottles of beer\n", 42)
>> 42 bottles of beer
19

This article is in no way a complete guide for extending/embedding python. For detailed explanation of above topics one might refer to:
http://www.linuxjournal.com/article/8497
https://docs.python.org/2/extending/
http://stackoverflow.com/questions/4455954/executing-and-reading-the-output-of-a-binary-in-python-bash-perl-script
http://stackoverflow.com/questions/2473655/how-to-make-a-call-to-an-executable-from-python-script
http://www.cyberciti.biz/faq/python-run-external-command-and-get-output/

Friday, March 21, 2014

Function overloading in python

Python doesn't do function overloading. This is a consequence of it being a loosely-typed language. Instead you can specify an unknown number of arguments and deal with their interpretation in the function logic.
There are a couple ways you can do this. You can specify specific optional arguments:
def func1(arg1, arg2=None):
    if arg2 != None:
        print "%s %s" % (arg1, arg2)
    else:
        print "%s" % (arg1)
Calling it we get:
>>> func1(1, 2)
1 2
Or you can specify an unknown number of unnamed arguments (i.e. arguments passed in an array):
def func2(arg1, *args):
    if args:
        for item in args:
            print item
    else:
        print arg1
Calling it we get:
>>> func2(1, 2, 3, 4, 5)
2
3
4
5
Or you can specify an unknown number of named arguments (i.e. arguments passed in a dictionary):
def func3(arg1, **args):
    if args:
        for k, v in args.items():
            print "%s %s" % (k, v)
    else:
        print arg1
Calling it we get:
>>> func3(1, arg2=2, arg3=3)
arg2 2
arg3 3
You can use these constructions to produce the behaviour you were looking for in overloading.

Thursday, November 21, 2013

Fatal error: Python.h: No such file or Directory

Fatal error: Python.h: No such file or Directory

When i tried to build mysql connector for python got the below error:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,4,'beta',4) -D__version__=1.2.4b4 -I/usr/include/mysql -I/usr/include/python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o -DBIG_JOINS=1 -fno-strict-aliasing -g
_mysql.c:29:20:
fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

Where can I find Python.h file and how do I fix this problem under Debian or Ubuntu Linux sever based system?

Python.h is nothing but a header file. It is used by gcc to build applications. You need to install a package called python-dev. This package includes header files, a static library and development tools for building Python modules, extending the Python interpreter or embedding Python in applications.

To install this package, enter:

sudo apt-get install python-dev

Saturday, September 7, 2013

How to install selenium on Windows?

Dependencies:

1. Install python 3.3
2. Add python.exe path in Environment variable
3. download setuptools from https://pypi.python.org/pypi/setuptools/0.7.4
    3.1.  extract downloaded tar
    3.2  go to extracted directory
    3.3  python ez_setup.py install

Install Selenium:
4. Download selenium from https://pypi.python.org/pypi/selenium
5. Extract downloaded tar file
6. Go to selenium extracted directory
7. python setup.py install


Thursday, January 20, 2011

How to convert your laptop into a Wireless HotSpot?

This article will help you convert your Windows XP and Vista basedlaptop in a Wireless . One you setup your laptop asHotSpot then you do not need a router, switch, hub, etc… to form a network, in fact other laptops or PC’s can connect or access the internet through this newly created HotSpot

What do I need to set a ?

  1. Active Cable Internet
  2. Laptop
  3. WiFi on Laptop
  4. Wired net connection

Once you have all the above things mentioned, you are all set to follow the steps:

How to setup on Windows Vista:

  1. Got to Control Panel
  2. Click on “Network and Sharing Centre”
  3. Under Tasks choose “Setup New Connection”
  4. Wireless “Ad-hoc network”
  5. Now type in a “Network Name”
  6. Choose “WEP” password if you don’t want to share the connection with the entire world
  7. Click “Internet connection sharing”
  8. Done!

Now a WiFi enabled machine will be able to connect to the network you have just setup with the WEP password

Lets setup a on a Windows XP machine:

  1. Go into Control panel
  2. Open “Network Connections”
  3. Right click on “Wireless Connection” and click on “Properties”
  4. Click on the “Wireless Networks” tab and select the check-box which says “Use Windows to configure my wireless network settings”
  5. Under “Prefered networks” click “Add”
  6. Type a name in “Network name” text box
  7. Select “Shared” from the options under “Network Authentication”
  8. Under Data encryption select “WEP” and type in your password in “Network Key” field and now click “OK”
  9. Now the new should show in the list of preferred networks.
  10. Click on the “Advance” button and select “Computer-to-computer (ad hoc) networks only” and press close
  11. Hit “OK”, we are done with the wireless connection properties

One more step, we have to change something in the wired connection setting so that it knows that it has to share connection with the computers accessing our .

  1. Go to local Area Connections, right click it and select properties
  2. Go to the “Advanced” tab and under Internet Connection Sharing check the box which says “Allow other network users to connect through this computer’s internet connection” and make sure the box under that is NOT selected so that no one can disable the network remotely.
  3. Hit “OK”

I hope this helps setting up a wireless connection without a WiFi router. Have FuN!

Wednesday, January 19, 2011

REST simplified...

A brillint explanation of REST by Joe Gregorio on xml.com.
http://www.xml.com/pub/a/2004/12/01/restful-web.html