Python - Different ways to import Python modules

 

In Python each file ended with extension .py is called a python module. these modules can be imported into your code in the below mentioned ways.

To keep it in short and sweet listed below with short explanation for the same.

source.py

def func1(num1):
    print(f"func1 - {num1}")

def func2(num2):
    print(f"func2  - {num2}")

I am trying to import source python module in to destination.py module. 

Method 1

import source
source.func1(10)
source.func2(20)

Method 2

import source as nickname
nickname.func1(100)
nickname.func2(200)

 Method 3:

from source import func1

func1(100)
func2(200) #NameError: name 'func2' is not defined

Method 4

from source import *

func1(100)
func2(200)

Thank You

Get Prajavaani and Kannadaprabha e-paper every day in Linux machine

Download Prajavaani 

First let us understand how the  Prajavaani e-paper has been uploaded by the news paper press in http://prajavaniepaper.com.

http://prajavaniepaper.com/pdf/$YEAR/$MONTH/$DAY/${REVERSE_DATE}${EDITION_LETTER}_${PAGE_NUMBER}100.pdf
$YEAR ==> YYYY (e.g 2013)
$MONTH==> 01 - 12
$DAY==>01 - 31

$REVERSE_DATE ==> YYYYMMDD (e.g 20130825)
$EDITION_LETTER ==> a - z
e.g
       a ==>Main Edition
       c ==> Classifieds
       g ==> Bhoomika
       h ==> Kreeda Puravani
       j ==> Sapthaahika Puravani
       n ==> PV Metro
       p ==> Siskshana
       z ==> Additional Page
Not all these editions will be printed on all days.  For example, Saapthaahika puravaani will be printed only on Sunday, Bhoomika is only on Saturday etc.. Hence we have to check for all these letters while downloading.

Here is the script which will download  all the pages of the corresponding editions and finally it will merge each pdf pages in to  one PDF file( pdftk needs to be installed in your machine).

Steps to follow:
              1. Copy the below code in to one file (/root/downloadpaper.sh)  and provide the executable permission for this. After this,  please provide write access to $LOGFILE for all  users.

              2. Add the  below line as a root user in /etc/rc.local file. This will enable to download the paper  in background, when you boot the machine. The script is having logic of checking whether it has already downloaded or not. If not it will download.

bash  /root/downloadpaper.sh `date +%d` `date +%m` `date +%Y`

=====================================
#!/bin/bash

# usage  $0 DD MM YYYY

DAY=$1 
MONTH=$2
YEAR=$3

DATE=$DAY$MONTH$YEAR
REVERSE_DATE=$YEAR$MONTH$DAY
DIRECTORY="$HOME/prajavani/$DATE"
LOGFILE="/var/log/newspapers.log" // provide write access to all users chmod 777

if [ -e "$DIRECTORY/$DATE.pdf" ]; then
        echo "$DATE prajavani paper is downloaded, hence exiting"       >> $LOGFILE
        echo "===========================================================\n" >> $LOGFILE
        exit
fi

exit_status=1
while [ "$exit_status" -ne 0 ]
do
        wget -q prajavaniepaper.com
        exit_status=$?
        sleep 5

done
rm -rf index.html
echo "Downloading prajavani paper of $1/$2/$3 date on `date +%d/%m/%Y`" >> $LOGFILE
mkdir -p $DIRECTORY
cd $DIRECTORY
INPUT=""

for letter in a b c d e f g h i j k l m n o p q r s t u v w x y z
do
        exit_status=0
        pageno=1
        while [ $exit_status -eq 0 ]
        do
                wget -q "http://prajavaniepaper.com/pdf/$YEAR/$MONTH/$DAY/${REVERSE_DATE}${letter}_$(printf %03d $pageno)100.pdf"
                exit_status=$?
                let pageno=$pageno+1
        done
        if [ $exit_status -ne 0 -a $pageno -gt 2 ]; then
                let "pageno = $pageno - 2"
                i=1;
                while [ $i -le $pageno ]
                do
                        INPUT="$INPUT ${REVERSE_DATE}${letter}_$(printf %03d $i)100.pdf"
                        let i=$i+1
                done
        fi
done

pdftk $INPUT output $DATE.pdf
rm $INPUT
echo "\nFinished merging of prajavani pages-- enjoy the reading :) " >> $LOGFILE
echo "===========================================================\n" >> $LOGFILE

KannadaPrabha

In Kannadaprabha paper, there is no separate letter for each edition. hence it is very simple to download. The way the date format is handled is different from prajavani paper. apart from everything is same. 

The  above  script is slightly modified  as below..

Steps to follow:
              1. Copy the below code in to one file (/root/downloadpaper.sh)  and provide the executable permission for this. After this,  please provide write access to $LOGFILE for all  users.

              2. Add the  below line as a root user in /etc/rc.local file. This will enable to download the paper  in background, when you boot the machine. The script is having logic of checking whether it has already downloaded or not. If not it will download.

bash  /root/downloadpaper.sh `date +%d` `date +%m` `date +%Y`

=============================================

#!/bin/bash

DAY=$1

MONTH=$2

YEAR=$3

if [ $MONTH -lt 10 ];then

        MONTH=`echo $MONTH | cut -b 2`

fi

if [ $DAY -lt 10 ];then

        DAY=`echo $DAY | cut -b 2`

fi

DATE=$DAY$MONTH$YEAR

DIRECTORY="$HOME/kannadaprabha/$DATE"

LOGFILE="/var/log/newspapers.log"

if [ -e "$DIRECTORY/$DATE.pdf" ]; then

        echo "$DATE kannadaprabha paper is downloaded, hence exiting"   >> $LOGFILE

        echo "===========================================================\n" >> $LOGFILE

        exit

fi

exit_status=1

while [ "$exit_status" -ne 0 ]

do

        wget -q archives.kannadaprabha.com

        exit_status=$?

        sleep 5

done

rm -rf index.html

echo "Downloading kannadaprabha paper of $1/$2/$3 date on `date +%d/%m/%Y`" >> $LOGFILE

mkdir -p $DIRECTORY

cd $DIRECTORY

exit_status=0

pageno=1

while [ $exit_status -eq 0 ]

do

        wget -q "http://archives.kannadaprabha.com/pdf/$DATE/$pageno.pdf"

        exit_status=$?

        let "pageno = $pageno + 1"

done

i=1;

INPUT=""

let "pageno = $pageno - 1"

echo "Finished downloading $pageno of kannadaprabha pages" >> $LOGFILE

while [ $i -lt $pageno ]

do

        INPUT="$INPUT $i.pdf"

        let "i = $i + 1"

done

pdftk $INPUT output $DATE.pdf

rm $INPUT

echo "\nFinished merging of kannadaprabha pages-- enjoy the reading :) " >> $LOGFILE

echo "===========================================================\n" >> $LOGFILE

Thanks,

Shivu

youtube: The uploader has not made this video available in your country.

   Sometimes, youtube link will not play in your country, but in other country it will play.  This is becausethe  up loader  has restricted the video to play in your country. In other wards they have allowed to access with in few countries.  In this case, you will get the  error as "The uploader has not made this video available in your country".

Depending on the gateway IP address, youtube application will figure out in which country you are in? And it will check whether up loader has provided  the access for this country. If yes it will play. Otherwise it will not play.

Don't worry. you can still play this video but with the cost of low speed.  The below steps explains in detail.

WINDOWS OS

1.  Please make sure the error is related to country specific.

2. Go to internet settings and LAN. check for the proxy settings. It will be blank  or your country specific proxy will be there.

3. Now go to "http://www.hidemyass.com/proxy-list/search-408711"

4. Check for most probable allowed country of the you tube link(trial and error method). Filter out that country. You may get multiple IP addresses and ports.

5.  Select long connection time IP address and port.

6. Configure  this in the proxy settings in internet settings.

7.  Re-launch  the  browser  and try with youtube link. It will work fine. sometimes connection will get dropped. we need to bare with that.

Linux OS

Because of  the issue of connection dropping with these free proxy servers, we can download the youtube video from youtube-dl application and then watch. I have not explored these tools in windows, But in lLnux I tried with youtube-dl  application which is youtube downloader.

1. First install youtube-dl package in Linux and make sure it is working properly. Sometime, it will throw some errors. Fix these errors and check that youtube-dl is working properly.

2. Follow the steps 3-5 of windows OS to get the proxy server IP  address  and  port.

3. Export this proxy in the terminal as

      %  export HTTP_PROXY="http://<selected IP address>:<port>/"

  e.g:
       %  export HTTP_PROXY="http://182.72.238.81:80/"

Please note that this will not set the proxy for the whole system. Instead it will set only for the terminal. This is very helpfull in case of downloading the youtube video.  Still you can set the configuration system wide. But as I told earlier it will slow down the network, because of all the requests will pass through this proxy and makes the internet slow.

4.  now you are set to download. You can run the youtube-dl command as  below:
   
      %  youtube-dl  <YOUTUBE URL>

Please note that  the connection will get dropped in between the download. hence I have written the script as a work around for this issue.

#!/bin/sh

export HTTP_PROXY="http://182.72.238.81:80/"

while read line
do

url=`echo $line`
echo "downloading $url"
exit_status=1
while [ $exit_status -ne 0 ];
do
youtube-dl  "$url"
exit_status=`echo $?`
done
done < links.txt

links.txt file is having the one youtube link per line.


Thanks
Shivu

How to get source code of a package?

  In this post, I will be explaining you to how to get the source code of  any package in Ubuntu.

Let us take one example and pull the source code. If you want to get the source code of  'ls' command in Ubuntu, First you need to know which package has bundled this command.

 1. First find the absolute path of the command wither by 'which'  command. If it didn't get the command name, then your PATH variable is not set properly.  No worries, search for the binary in your machine with find command.

      %which ls

         OR

     % find / ls 2>/dev/null

Let us say we got '/bin/ls' as output.

2. Now get the package name which bundled this command.

    %dpkg -S /bin/ls

    coreutils: /bin/ls

3. Coreutils is the package which bundled /bin/ls command.

Before going to pull the source of this package,  configure the apt by uncomment all the lines starts with 'deb-src' in /etc/apt/sources.list file as a root user. After doing this run apt-get update.

 

   %apt-get update

4. Create a directory where root has write permission. and  go to that directory. run  below command to get the source code of the package.

  %apt-get source coreutils

5. You will get the below files as part of this download. here we will be having 3 files and 1 directory.

            coreutils-8.20 ---> Directory which is extracted code of   coreutils_8.20.orig.tar.gz.

            coreutils_8.20-3ubuntu5.dsc ---> Description of the package, dependency list and signature.

            coreutils_8.20-3ubuntu5.diff.gz  ---> Patch, which enables the source code build on ubuntu.

            coreutils_8.20.orig.tar.gz --->  Original GPL code

**** you can find the source code for ls command in 'coreutils-8.20/src/ls.c' file. ****

 6. Applying patch to the source code.

 First extract the gzipped diff  file  and then apply patch to source code. 

  %gunzip coreutils_8.20-3ubuntu5.diff.gz

  % cd coreutils-8.20

  %patch -p1 < ../coreutils_8.20-3ubuntu5.diff 

Now you can build the source code and install your own binaries in your Ubuntu machine.

Thanks

Shivu

Hack password of PDF file in Ubuntu

  Every human being is tend to forget something important. This is the gift given by god to make space for new data in our brain(ROM and RAM).    Good example of forgetting the things are passwords of bank login, documents password which we only set etc...

Here  I am explaining how to crack the password of the pdf file in UBUNTU.

1. Install pdfcrack debian package from the apt-get command.

%sudo apt-get install pdfcrack

This package is available even for fedora, Centos  etc...

2. Run the below command.

%pdfcrack -f <filename> -s

 It will take long time to crack the password.
To improve the performance and to reduce the time taken to crack the password you can use options.
         If you know the number of characters of the password, then you can use -n and -m options.
                     -n minimum characters
                     -m maximum characters
        If we know the password  length is 5 or 6 characters, then we can  provide options as -n 5 -m 6
        If we know the possible password characters, then we can use -c option.  Most of the bank statements passwords will be in numbers, Hence we can use  -c 0123456789

3.  Here is the example, where the date and month of birth as a password. (01Jan or jan01)...

 %pdfcrack -f pdfcracktest.pdf -n 5 -m 5 -c 0123456789janfebmaraprmayjunjulaugsepoctnovdec -s

       With "09jan" password  I could able to open the document pdfcracktest.pdf.

Thanks
Shivu

Trace process specific kernel functions with ftrace

   In my previous blog, We have learnt  how to setup ftrace environment and function level tracing . In this post we will learn on how to trace the kernel function specific to a particular process and function filter.

1. trace kernel calls specific to process

As we know we have /sys/kernel/debug/tracing directory. In this we have set_ftrace_pid file.

•   First we need to make sure tracing_on is not enabled, by echoing 0 to tracing_on.
•   Disable all tracers by echoing 'nop' to current_tracer.
•   echo the pid of the process which we have to trace to set_ftrace_pid file. (say pid of ntpd, that is 2444)
•   Enable function or function_graph tracer by echoing corresponding tracing  in to current_tracer file.
•   Enable tracer by echoing 1 to tracing_on.
•   Now if you run the cat trace you will find the kernel function trace which are invoked by that process.

•  To disable this tracing  you need to do the reverse order. first disable tracing_on, then disable function tracing, and then at last echo nothing to set_ftrace_pid (to disable) as below.

             %echo > set_ftrace_pid

you can verify whether no pid has been written in to set_ftrace_pid file by opening the same.
I tried to echo -1 as documented, but it fails. Hence I found this workaround.

2. To trace specific functions:

    First we need to check whether the function which we are interested is in 'available_filter_functions'.

If not you can you can browse the kernel code to get which function is invoking the function, which you are interested in. So that you can search that function in this available_filter_functions list. 

Once you get that the function in the available_filter_functions file, echo that function to set_ftrace_filter file.  the process is very similar to the above one, except the instead of pid, we will echo the function to set_ftrace_filter. Make sure "no pid" phrase in set_ftrace_pid file.

 You can specify just one function or multiple functions in set_ftrace_filter file (one function per line).

Here in the below example I have selected ext4_write_inode and ext4_inode_table functions.
The below tracing filtered out  only those functions.

you can enable function graph  instead of function in the current_tracer.

To disable the  this tracing, we need to disable tracing_on,  current_tracer and then set_ftrace_filter in order. To disable set_ftrace_filter we can echo null to set_ftrace_filter.

            % echo > set_ftrace_filter

Thanks
Shivu

Linux kernel debug with ftrace

   
    Ftrace is the lightweight and flexible function level debugger. It can be used as data collection, debugging and performance tuning on Linux kernel. It does not need to recompile the kernel code. With just existing kernel code we can trace the kernel level function calls flow.   The below are the steps to to setup ftrace and how to use this.

1. first we need to make sure the debugfs is supported in the kernel.  This can be find in /proc/filesystems file. In my desktop kernel it has been enabled.

2. Now we need to mount the debugfs. Please verify whether this has been mounted by running mount command.

If you are not finding this entry, you can mount this by the below command. 

%mount -t debugfs none /sys/kernel/debug

**** Instead of /sys/kernel/debug, you can use your own directory while mounting. ****

3. Now you are all set to go. After you mounted you will find tracing directory, which has so many files as listed below. 

4.  To know the supported tracers, cat the file available_tracers.

5. As this post is about ftrace, I will concentrate on  function and function_graph tracers.

  To enable trace, as a root user echo 1 to tracing_on file as below:

 %echo 1 > tracing_on

Now  select which tracers need to be enabled. Let us  say we need  function trace. For this we need to echo function to current_tracer file as below:

%echo function > current_tracer

To view the trace information, we need to cat  or vi the trace file.  The trace file is a just reading the ring buffer  and display to the user.  The content can be still viewed even after disabling the  trace by echoing 0 to tracing on file.

%echo 0 > tracing_on. 

But if you  remove the  function trace from the current trace file, then all the ring buffer data will get erased. because of this, you can not view the trace information.

The above trace output explains all the the kernel calls with on which CPU it is running. The first column is showing the process name and corresponding PID, to which the kernel calls invoked.

The fourth column talks about the time spent from the boot time.

6. To disable trace logging,  just echo 0 to tracing_on file. As explained above this will not erase the ring buffer content instead it will stop logging data to ring buffer. Hence we can still view the trace file(ring buffer)  content.

7.  If we need to get the call graph in heirarchical manner,  we have to echo function_graph to current_tracer file.

%echo function_graph > current_trace

The function_graph trace is showing the time spent in each kernel function. The time will be displayed in the second column corresponds to each leaf function.  If a  function is calling nested functions, then the accumulated time will be displayed, while exiting from that function.  If the time taken in a function is exceeding  10 Microseconds, then it will be denoted as '+'. if it is taking more than 100 microseconds, then it will be denoted as '!'.

In my next post will explain about how to attach a process  and how to filter out the functions in the trace output.

Thanks

-Shivu