Essential Python

Essential Python

Python in VSCode is documented here

Try not to use a separate IDE for each technology!!!

Anaconda seem to be around 600MB and seem to have lots of bells and whistles such as R and data science stuff.

For starters I went with the windows python app in windows store at 60MB I believe.

When installed and run it seem to open a python 3.7.4 interactive shell.

The shell seem to suggest a tutorial

The standard library is documented here

High level literal data structures such as lists, tuples (tables/rows/columns)

interpreted, dynamic, Scripting benefits

Object oriented

open for modules and new packages

Python packages called pypi (python package index) is documented here

w3schools and python

len() for example is a built in function in python: The other are here

Built-in functions in Python

Search for: Built-in functions in Python


# This is a comment

#This is a hello world in python
print ("hello world")

#A partial comment
msg1 = "Partial comment" #Partial comment

# Common
# Escaped strings with # single quotes and double quotes

#************************************
#Multiline strings
#End of lines are preserved unless removed with an escape \ 
#************************************
multilineString = """First line
Second line
third line
"""
print (multilineString)

#************************************
#Glued strings
#No need for + if next to each other
#************************************
msg2 = "test" + "another test"
msg3 = "test" "another test"
print (msg2)
print (msg3)

#You have to put them in brackets to make the following work
msg4 = ("test"
    "another test string"
    "and one more string")

print (msg4)



#************************************
#Agreeing with escapes
#************************************
msg5 = r"c:\rootdir\otherdir"
msg6 = "c:\\rootdir\\otherdir"

#************************************
#Comparing strings
#************************************

if msg5 == msg6:
    print ("yes they are same")
else:
        print("No, they are not the same")

print (msg5)
print (msg6)

#************************************
#Finding strings using "in" key word
#************************************
available1 = r"c:\root" in msg5
available2 = "c:\root" in msg5
print (available1)
print (available2)

#************************************
#Subscripting
#************************************ 
msg5 = "0123456789"
msg7 = msg5[:4] #Get 0 to 3
print (msg7)

#Notice how it gets the 5th character
#as the array is 0 based
msg8 = msg5[4:8] #4,5,6,7
print (msg8)    
c7th = msg5[6]
print (c7th) #You will get 6

#************************************
#Length
# len() is what is called a Built in function
#************************************ 
print (len(msg5)) #should be 10

#************************************
#Cases: Title, Upper, Lower
#************************************ 
msg9 = "how ABOUT This sEntence"
print (msg9.title()) #How About This Sentence

Python is an object oriented programming language.

Almost everything in Python is an object, with its properties and methods.

They are first class

don't need to be in an object

default args supported

indented for completion

types are inferred and not specified

arguments can be in sequence or named


def my_function(food):
  for x in food:
    print(x)

fruits = ["apple", "banana", "cherry"]

my_function(fruits)

Arbitrary arguments with a * in python

Search for: Arbitrary arguments with a * in python


Lists: [a,b,c] #Changeable
Tuples: (a,b,c) #Unchangeable
Sets: {a,b,c} #unordered, unindexed
Dictionaries: {a:b, c:d} #uordered, changeable, indexed

How to reference a self object in python?

Search for: How to reference a self object in python?

Read through this

import inspect:Search On Web

import logging:Search On Web

import traceback:Search On Web


#***********************************
#Function: printCollected
#***********************************


def printBeginMsg(msg):
    print ("*****************************")
    print ("*" + msg)
    print ("*****************************")

def printEndMsg(msg):
    print ("*****************************")
    print ("*" + msg)
    print ("*****************************")

def printCollected(msg, collectedRDD):
    printBeginMsg(msg)
    for item in collectedRDD:
        print (item)
    printEndMsg(msg)

#***********************************
#End of function
#***********************************

How do I know the type of an object in python?

Search for: How do I know the type of an object in python?


#Print type
print (type(someobjectRef))

#examine type

class T

isinstance(object, T)


# you cannot do this
print("blah " + type(o))

# what you have to do

type1 = type(o)
type2 = type(o)
print("blah type1 {} type2 {} etc ".format(type1,type2))


# r before a string stands for raw
# This gives an error however
dataDirectory = r'C:\satya\data\code\pyspark\'
# SyntaxError: EOL while scanning string literal

#this is ok
dataDirectory = r'C:\satya\data\code\pyspark'

Online Book: This python text book is a good online quick reference


#Some constants
dataDirectory = r'C:\satya\data\code\pyspark'

processedDir = dataDirectory     + r"\data\mp_scada_data\processed"

inputWuActualWeatherForAWeek =     dataDirectory     + r"\data\mp_scada_data\wu_actual\All Data.csv"


def printVar(msg, var1):
    formatMsg = msg + ": {}"
    finalMsg = formatMsg.format(var1)
    print(finalMsg)

printVar ("Data directory", dataDirectory)
printVar ("inputWuActualWeatherForAWeek",inputWuActualWeatherForAWeek)

#You can also do this instead
print ("Data directory: ", dataDirectory)
print ("inputWuActualWeatherForAWeek: ",inputWuActualWeatherForAWeek)

List comprehensions in python

Search for: List comprehensions in python


if len(mylist) < 10:
    print (mylist)
else:
    print ("List too big")

Understand enumerate in python comprehensions

Search for: Understand enumerate in python comprehensions


mylist = [line for idx, line in enumerate(inputFileObj) if idx < 5]

print ("*********")
print (len(mylist))
print (mylist[0])

if len(mylist) < 10:
    print (mylist)
else:
    print ("List too big")
    print (len(mylist))


true: "dddd"
false: "" //empty

//All false because they are empty
//if not then true
[]
{}
()

false: 0
true: 1
false: None


Whats up with :
     Starts a block of code
     In #python
     Examples:
           If ... :
           for ... :
     :)


if (x > y)

is ok to say

if x > y


if (a == 1 and
   b == 2 and
   c == blah ) :
   other stuff

where brackets are extending a line. there may be nuances as brackets can define a tuple as well!! 


#unpack a list or tuple etc
a,b,c,d,e = "hello"

then
a = h
b = e
c= l
d = l
e = o

working with Date time in python

Search for: working with Date time in python


from datetime import datetime

someInstance = datetime.fromisoformat("2019-09-01 00:05:00.000")

print(someInstance)

#prints: 2019-09-01 00:05:00


from datetime import datetime

someInstance = datetime.fromisoformat(
    "2019-09-01 00:05:00.000"
    )
    
print(someInstance)


from datetime import datetime

someInstance = datetime.fromisoformat(
    "2019-09-01 00:05:00.000"
    )

print(someInstance)
print(someInstance.year)
print(someInstance.month)
print(someInstance.day)

string formatting integers in python

Search for: string formatting integers in python


from datetime import datetime

sampleTimeInstanceString = "2019-09-01 00:05:00.000"
someInstance = datetime.fromisoformat(
    sampleTimeInstanceString
    )

print(someInstance)
print(someInstance.year)
print(someInstance.month)
print(someInstance.day)

def getFilename(datetimeInstanceString):
    fileBasename = "somebase-filename-for-"
    #i - instance
    i = datetime.fromisoformat(datetimeInstanceString)

    #see the formating options for padding 
    #integers with 0s
    filePostFix = (
        "{}{:02d}{:02d}{:02d}"
        .format(i.year, i.month,i.day, i.hour)
    )
    return fileBasename + filePostFix + ".csv"

print (getFilename(sampleTimeInstanceString))
# will pring the following
# somebase-filename-for-2019090100.csv


filePostFix = (
        "{}{:02d}{:02d}{:02d}"
        .format(i.year, i.month,i.day, i.hour)
    )

Working with directories in python

Search for: Working with directories in python

Here is os.path in the standard library of python

There doesn't seem to be an explicit structure like a "directory"

File and directory access in python docs

working with files and directories in python

Basic intro to creating classes

How to raise exceptions in Python

Search for: How to raise exceptions in Python

Here is the raise keyword in python lang reference

class documentation in lang reference

calling other methods for __init__ in python

Search for: calling other methods for __init__ in python

See this discussion in SOF


#calling form init
#notice "self."
self.f1()

#define it 
def f1(self):

How to delete files from a folder

Search for: How to delete files from a folder

How can I delete the contents of a local folder in Python?

Search for: How can I delete the contents of a local folder in Python?

at SOF: How can I delete the contents of a local folder in Python?


os
shutil
os.path
os.listdir
shutil.rmtree
os.unlink
glob.glob
os.remove(file)


import os
from os import path
import shutil

def _emptyFilesetDirectory(self, fileSetDirectory):

        for item in os.listdir(fileSetDirectory):

            #item is just a name, not the full path
            itemFullpath = path.join(fileSetDirectory,item)


            if path.isfile(itemFullpath):
                os.remove(itemFullpath)
                continue

            if path.isdir(itemFullpath):
                shutil.rmtree(itemFullpath)
                continue

How to put imports on multiple lines in python

Search for: How to put imports on multiple lines in python


from SimpleXMLRPCServer import SimpleXMLRPCServer, \ 
            SimpleXMLRPCRequestHandler, \ 
            CGIXMLRPCRequestHandler, \ 
            resolve_dotted_attribute

Or

from SimpleXMLRPCServer import (SimpleXMLRPCServer,
                                SimpleXMLRPCRequestHandler,
                                CGIXMLRPCRequestHandler,
                                resolve_dotted_attribute)

These are called multi-line imports in python

Search for: These are called multi-line imports in python

Multi-line imports are documented here


from outputfilesetdirectory import     (getTheHour, 
    FileSetDirectory, 
    getFilenameByHour)


from outputfilesetdirectory import (
    getTheHour,
    FileSetDirectory,
    getFilenameByHour 
)

File IO is documented here

python partially assign variables from a list

Search for: python partially assign variables from a list

Partial list unpack in Python: SOF

* operator multiple assignment syntax in python 3

Search for: * operator multiple assignment syntax in python 3

Multiple assignment and tuple unpacking improve Python code readability: article


def getFirstField(aRowWithCommaSeparatedFields):
     
    #Get some type help by saying it is a string
    line : str  = aRowWithCommaSeparatedFields

    #line is a set of fields
    #First field is an iso time format
    #2019-09-01 00:05:00.000
    #Many other comma separated fields

    #Using type help you can discover .partition method
    timeField, *rest = line2.partition(",")

    print(timeField)
    print(line)

with statement to open and write to a file in python

Search for: with statement to open and write to a file in python

with statement in python

Search for: with statement in python

with statement is documented here


with someObject as x:
  suite of statements
  that use x

could be equivalent to

x = someobject.enter(self)
execute suite
someobject.exit(self, exc_type, exc_value, traceback)

1. if enter succeeds, it is guaranteed that exit is called.

2. if no exception, then None objects are passed to exit

3. if exception, then those exception objects are passed in

4. exit can return true to stop the propagation of exception otherwise the exception is passed on


with open(filename, filemode) as myfile:
  myfile.write(...)

The myfile object will be guaranteed to be closed irrespective of the exception or success

Working with context managers in python

Search for: Working with context managers in python


//Ensure this
Do some thing before

//then do this
Do code

//Ensure this even with failures
Do something after

Context Managers are documented in GitHub book PythonTips book by Yasoob

Enum is documented here

Enums in python

Search for: Enums in python

Idea of using Enum for file modes open() in python is discussed here

Using Enum for file modes open() in python

Search for: Using Enum for file modes open() in python


class FileMode:
    #Some popular modes
    #Reads
    r = "r"
    read = r

    #writes    
    w = "w"
    write = w #overwrite

    #append
    a = "a"
    append = a

    #Attach this where needed
    binary = "b"



#Notice the FileMode. for intellisense

with open(filepath, FileMode.w) as file:
        for line in actualWeatherByTheHourList:
            file.write(line)


[x for x in list if x % 2 == 0]

or

[f2(x) for x in list if f(x) == true]

same as

select x // before for
from table //for part
where f(x) == true //if part

type hints in python

Search for: type hints in python

Type hints are documented here


def greeting(name: str) -> str:
    return 'Hello ' + name

Multi-line comment in python

Search for: Multi-line comment in python

PYthon does not have multi-line comments.

  1. Install the ubiquitous MS python extension
  2. Go to command: Ctrl-shift-p
  3. Search for "Preferences: work space settings"
  4. Then there search for python.analysis.typeCheckingMode
  5. Set it to "strict"

#In your function give a type hint as below
def understandResponse(response: requests.Response):
    response.

After the "." tupe ctrl-space, and you will ONLY those methods specific to that object. 


lib1 imports lib2
App import lib1 and also lib2

In other words

lib1 -> lib2
App -> lib1, lib2

Cascading imports are common in Python


def localTest():
    print ("Starting local test")
    print ("End local test")

if __name__ == '__main__':
    localTest()


try:
    # Code that may raise an exception
    num = int(input("Enter a number: "))
    result = 10 / num  # This line may raise a ZeroDivisionError if num is 0
    print("Result:", result)

except ZeroDivisionError:
    # Handle the specific exception (ZeroDivisionError)
    print("Error: Cannot divide by zero")

except ValueError:
    # Handle another specific exception (ValueError)
    print("Error: Invalid input. Please enter a valid number")

except Exception as e:
    # Handle any other exception not caught by the specific except blocks
    print(f"An error occurred: {e}")

else:
    # Optional else block that executes if no exceptions occurred
    print("No exceptions occurred")

finally:
    # Optional finally block that always executes, whether there's an exception or not
    print("Execution finished")


class Color(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

# then
if (Color.Red == 1) etc.


main_string = "Hello, world!"
sub_string = "world"

if sub_string in main_string:
    print(f"'{sub_string}' is present in '{main_string}'")
else:
    print(f"'{sub_string}' is not present in '{main_string}'")