Loading classes dynamically
satya - 3/9/2024, 9:53:21 PM
What does a fully qualified classname look like in python
[MyClass]
class_name = "my_package.my_module.MyClass"
satya - 3/9/2024, 10:23:56 PM
Consider the following structure
baselib
__init__.py
file1.py
configlib
__init__.py
file2.py
....say file2 has a class name called "File2Class"
clientlib
__init__.py
file3.py
satya - 3/9/2024, 10:24:46 PM
Fully qualified name for File2Class is
configlib.file2.File2Class
# that is
#pkg.module.classname
satya - 3/9/2024, 10:25:36 PM
In this example the package names in this project are
- baselib
- configlib
- clientlib
satya - 3/9/2024, 10:26:11 PM
And the modules are (of course in their respective packages)
- file1
- file2
- file3
satya - 3/9/2024, 10:31:57 PM
Loading a module dynamically: importlib
import importlib
module_name = "mymodule"
mymodule = importlib.import_module(module_name)
satya - 3/9/2024, 10:33:09 PM
Checking for import errors dynamic or otherwise: ModuleNotFoundError
try:
import optionalmodule
except ModuleNotFoundError:
print("optionalmodule is not installed.")
satya - 3/9/2024, 10:33:58 PM
To see if a module is loaded: via sys.modules
import sys
if "mymodule" in sys.modules:
print("mymodule is already loaded.")
else:
print("mymodule is not loaded yet.")
satya - 3/9/2024, 11:04:33 PM
Example of loading classes dynamically
import importlib
from typing import Any, Type
def load_class(full_class_name: str) -> Any:
module_name, class_name = full_class_name.rsplit('.', 1)
module = importlib.import_module(module_name)
cls: Type[Any] = getattr(module, class_name)
return cls()
def load_class_with_default(full_class_name: str, default_object: Any) -> Any:
try:
return load_class(full_class_name)
except ModuleNotFoundError as e:
log.logException(f"Module not found for {full_class_name}", e)
except AttributeError as e:
log.logException(f"Class not found for {full_class_name}", e)
except Exception as e:
log.logException(f"Failed to creae {full_class_name} ", e)
return default_object
satya - 3/30/2024, 3:22:22 PM
Calling a constructor dynamically
def _createObjWithInit(cls: Type[Any], args: dict[str,Any]) -> Any:
return cls(**args)
satya - 3/30/2024, 3:24:57 PM
Observations
- The ** operator takes a dictionary and allows the init method to parse its arguments based on the names of the arguments
- The **args expects a dictionary for "args"
- where as "*args" expects a tuple and a positional way of passing args
- The **args works even if the class has positional args only as long as their names match
- This covers for default args as well
satya - 3/30/2024, 3:27:13 PM
kwargs and the dictionary that gets passed and its contents
- If the dictionary has a key whose name is not a listed argument it will result in an error
- Unless the class init method has a **kwargs as one of its inputs
- In that case the additional args will be passed as the "kwargs" into the init function
satya - 3/30/2024, 3:28:08 PM
Consider this class
class TestClass():
arg1: str
arg2: int
defaultarg: str
def __init__(self, arg1: str
,arg2: int,
defaultarg: str = "charge"
, **kwargs: dict[str, Any]
):
self.arg1 = arg1
self.arg2 = arg2
self.defaultarg = defaultarg
log.info("Additional arguments")
log.prettyPrintDictionary(kwargs)
satya - 3/30/2024, 3:29:29 PM
This demonstrates the following
- positional arguments: arg1, arg2
- default arguments: defaultarg
- A dictionary of variable arguments: **kwargs
satya - 3/30/2024, 3:31:55 PM
Lets see how this class can be instantiated dynamically in a variety of ways
"""
*************************************************
* Creates an object by calling the "init" method
* Exepects to know the arguments to pass to the init method
* Useful for loadign data classes from configuration files
* Can also be used for factory classes where the constructor is virtualized
*************************************************
"""
def _testCreateObjwithInit():
cls = load_class("aspire_tinyapp.baselib.factoryutils.TestClass")
# case: with kwargs as well
# you cannot pass additional args if the class doesn't have kwargs
# additional args can be just key value pairs and not a dictionary necesssarily
# The unused params will go as kwargs (i think. tested. that is correct)
#argDict = {"arg1": "hello", "arg2": 5, "additional_org": "additional_stuff", "and more": "stuff"}
# case: without kwargs in the class
# it will fail if you pass additional args
# only the args that are expected will need to be passed
#argDict = {"arg1": "hello", "arg2": 5, "additional_org": "additional_stuff", "and more": "stuff"}
#case with the default args
# default org will override the default value in the class
# additional args passed as kwargs
# if kwargs are ommitted in the class, passing them will result in an error
#
argDict = {"arg1": "hello", "arg2": 5,
"defaultarg" : "default org",
"additional_org": "additional_stuff",
"and more": "stuff"}
log.ph1("Reflection instantiation test")
obj = _createObjWithInit(cls, argDict)
log.info("Object field values")
log.prettyPrintObject(obj)
log.ph1("Direct instantiation test")
obj = TestClass("hello", 1)
log.prettyPrintObject(obj)
satya - 3/30/2024, 3:33:01 PM
case: without kwargs in the class
- It will fail if you pass additional args in the input dictionary
- Only the args that are expected by the init method will need to be passed
satya - 3/30/2024, 3:35:06 PM
case: with kwargs as well
- You cannot pass additional args if the class doesn't have kwargs
- Additional args can be just key value pairs and not a dictionary necessarily when you are passing them
- In other words you don't have to pass a dictionary
- The unused params by the init method will go as kwargs to the init method
satya - 3/30/2024, 3:37:24 PM
case with the default args
- default org will override the default value in the class
- either you can pass it or not
- additional args passed as kwargs
- if kwargs are ommitted in the class, passing them will result in an error
satya - 3/30/2024, 4:21:04 PM
Lets delve into figuring out the parameters of a constructor dynamically
def _getConstructorParams(cls):
init_signature = inspect.signature(cls.__init__)
params = []
for name, param in init_signature.parameters.items():
if name == 'self':
continue
param_type = param.annotation.__name__ if param.annotation is not param.empty else 'No Type Info'
has_default = param.default is not param.empty
param = SimpleParam(name, param_type, has_default)
params.append(param)
return params
satya - 3/30/2024, 4:21:57 PM
Essentials, for codegen
- Should return a list of parameters
- each parameter should be a tuple of (name, type, default)
- Should handle the case where there are no parameters
- should return an empty list in that case
- I just need to know if the param has a default or not
- I don't need "self" as a parameter
- I also just need the classtype as a "string" and not the actual class
satya - 3/30/2024, 4:22:21 PM
The SimpleParam class I needed
class SimpleParam():
def __init__(self, name: str, type: str, default: bool):
self.name = name
self.type = type
self.default = default
def __repr__(self) -> str:
return f"Name: {self.name}, Type: {self.type}, Default: {self.default}"
satya - 3/30/2024, 4:22:50 PM
Testing it
def _testGetConstructorParams():
cls = load_class("aspire_tinyapp.baselib.factoryutils.TestClass")
params = _getConstructorParams(cls)
log.info("Constructor parameters")
log.info(f"{params}")
satya - 3/30/2024, 4:23:30 PM
Prints out
info:
[Name: arg1, Type: str, Default: False,
Name: arg2, Type: int, Default: False,
Name: defaultarg, Type: str, Default: True,
Name: kwargs, Type: dict, Default: False]