Thursday, November 16, 2017

Python Defensive Iteration

This is the item 12 explained in the “Effective Python”, by Brett Slatkin. Generators are the best save of out of memory problem. Here the problem,

class StringToList(object):
    def __init__(self, text):
        self.text = text

    def __iter__(self):
        for x in self.text:
            yield x

words = StringToList('Hello')

# case 1
it = iter(words)
it2 =iter(it)
print(it, it2) #(<generator object __iter__ at 0x10816d7d0>, <generator object __iter__ at 0x10816d7d0>)
next(it) # 'H'
next(it2) # 'e'

The problem is it and it2 pointing to the same instance and iteration is not as expected (‘it2’ give a next element as ‘e’ instead ‘H’). To overcome this problem, author has suggested the following solution which is applicable to set and dict as well. In the case 1, same container is used. But in the case 2, different containsers.

# case 2
it = iter(words)
it2 =iter(words)
print(it, it2) # (<generator object __iter__ at 0x10816d550>, <generator object __iter__ at 0x10816d320>)
next(it) # 'H'
next(it2) # 'H'

In the above code, diffrent instaces because two diffrent containers. To defence from the case 1, need to use the following defence and avoid:

if it is it2:
    raise TypeError('Same container error')
else:
    print(next(it)) # 'H'
    print(next(it2)) # 'H'

That will promote the case 2 style of programming.

Monday, August 28, 2017

PyPI hosting on AWS S3

Create a web hosting in the AWS S3

It is a common task to create a web hosting in the S3 bucket. However, you have to change the permission to access the bucket via http protocol. For that you can right click the dist folder and select the Make public from the drop down menu.

Create Distribution package

Create a directory for distribution package source for example ojservice. Your folder structure is as follows:

ojsevice+
        |
        +index.html
        |
        +error.html
        |
        +LICENSE.txt
        |
        +README.txt
        |
        +setup.py
        |
        +ojservice+
                  |
                  +__init__.py
                  |
                  +helloservice.py

In the above structure, index.html and error.html are two files given in the AWS S3 host configuration under the bucket’s Static web hosting : respectively index and error documents.

Here the helloservice.py:

def hello(name):
    return 'Hello {}'.format(name)

In addition to that you have to install the package pip2pi

pip install pip2pi

Create setup.py file for the distribution:

from distutils.core import setup

setup(
    name='ojservice',
    version='0.1',
    packages=['ojservice',],
    license='Coreative Commons Attribution-Noncommercial-Share Alike license',
    long_description=open('README.txt').read(),
)

Create Distribution package which target is dist folder under the parent directory

python setup.py sdist

run the following command to create the package hierarchy:

dir2pi dist

Sync the folder to the s3 bucket for example: ojpy

aws s3 sync dist/simple s3://ojpy

Install the package using pip

Setup the client application to import and test the pip distribution package. Virtual environment is the best.

mkdir ojtest
cd ojtest

#setup virtual env
virtualenv ~/.virtualenvs/ojtest

#activate the enviroment
source ~/.virtualenvs/ojtest/bin/activate

Here the command

pip install --index-url=http://<s3 host>/ojservice ojservice==0.1 --trusted-host <s3 host>

Here the example

pip install --index-url=http://ojpy.s3-website-ap-southeast-2.amazonaws.com/ojservice/ ojservice==0.1 --trusted-host ojpy.s3-website-ap-southeast-2.amazonaws.com

Here the code snippet to import from the distributed package

from ojservice import helloservice
print(helloservice.hello('OJ'))

Tuesday, May 02, 2017

MSSQL on Mac

I was inspired to write this simple blog after reading the Microsoft Migrate a SQL Server database from Windows to Linux using backup and restore.This article about how to restore the AdventureWork on Linux, but this blog about MSSQL server 2017 on Mac OS Sierra.

MSSQL available as a docker image to install on Mac. If you have Docker installed in your Mac, simple to install with two commands as shown in the http://aka.ms/sqldev site:

docker pull microsoft/mssql-server-linux
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=yourStrong(!)Password' -p 1433:1433 -d microsoft/mssql-server-linux

After the docker installation, login to the doker:

docker exec -i -t <CONTAINER ID> /bin/bash

Create a new folder “backup” in the “/var/opt/mssql”. If you are not admin run the command sudo su to login as super user.

cd /var/opt/mssql/
mkdir backup

Your work as admin in the docker is finished. Download the AdventureWork from https://msftdbprodsamples.codeplex.com/downloads/get/880661 to your local machine and unzip. Then copy to Docker instance

docker cp AdventureWorks2014.bak <CONTAINER ID>:/var/opt/mssql/backup

If you don’t have sqlcmd installed in your host machine follow the article to install :

brew install mssql-tools

In your host machine use the sqlcmd tool to login to the MSSQL server:

sqlcmd -S 127.0.0.1 -U SA -P 'yourStrong(!)Password'

Following the link https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-migrate-restore-database

Restore the backup file:

RESTORE DATABASE AdventureWorks
FROM DISK = '/var/opt/mssql/backup/AdventureWorks2014.bak'
WITH MOVE 'AdventureWorks2014_Data' TO '/var/opt/mssql/data/AdventureWorks2014_Data.mdf',
MOVE 'AdventureWorks2014_Log' TO '/var/opt/mssql/data/AdventureWorks2014_Log.ldf'
GO

Now you are ready to go. I am using Visual Studio Code and the mssql plugin to access this database.