Object Oriented Programming in Python
When developing Python code for data processing we eventually encounter the need to use object oriented programming. There is nothing inherently difficult about object oriented programming, except that it requires a somewhat convoluted thinking process. We illustrate the idea here with a simple example.
Least Squares
We are given observation vectors $\bby\in\reals^m$ and matrices $\bbA \in \reals^{m\times n}$ and we are asked to find the vector $\bbx\in\reals^n$ that solves the least squares problem
\begin{equation}\label{eqn_linear}
\bbx^* = \arg\min_\bbx \frac{1}{2}\| \bby – \bbA \bbx \|^2 .
\end{equation}
The solution to this problem is to make $\bbx^* =(\bbA^T\bbA)^{-1}\bbA^T\bby$, which we can verify is true by taking the gradient of \eqref{eqn_linear} and setting it to zero.
In order to solve least squares we write a function that takes $\bbA$ and $\bby$ as inputs, and computes the solution $\bbx^* =(\bbA^T\bbA)^{-1}\bbA^T\bby$ which it returns as an output:
class LinearFunction:
import numpy as np
def ls(A, y):
AT = np.transpose(A) # Compute $\bbA^T$}
ATA = np.matmul(A,AT) # Compute $\bbA^T\bbA$}
iATA = np.linalg.inv(ATA) # Compute $(\bbA^T\bbA)^{-1}$}
iATAAT = np.matmul(AT,iATA) # Compute $(\bbA^T\bbA)^{-1}\bbA^T$}
x = np.matmul(y,iATAAT) # Compute $(\bbA^T\bbA)^{-1}\bbA^T\bby$}
return x
Once we have programmed this function we can use it to solve least squares for any matrix-vector pair that we are given. For instance, consider a least squares problem in which the number of columns of $\p{A}$ is $n=3$ and the number of rows of $\p{A}$, which is also the number of columns of $\p{y}$, is $m=13$. The entries of both are random, drawn from a Gaussian distribution with zero mean and variance 1. The following code generates the matrix $\p{A}$ and the vector $\p{y}$ according to this specification
class LinearFunction: m = 13 n = 3 A = np.random.normal(0, 1, size=(n, m)) y = np.random.normal(0, 1, size=(1, m))
And we can solve the corresponding least squares problem by invoking the function $\p{ls}$:
class LinearFunction: x = ls(A, y)
If we want to solve least squares for a different matrix and different vector, we call the function $\p{ls}$ with different arguments:
class LinearFunction: AnotherA = np.random.normal(0, 1, size=(n, m)) Anothery = np.random.normal(0, 1, size=(1, m)) x = ls(AnotherA, Anothery)
There is nothing wrong with using this approach to code the solution of a least squares problem. But in data science circles developers are more accustomed to object oriented programming. This requires that we create objects that instantiate classes where we specify the operations that are to be performed. This results in code that can look weird and complicated but that, most argue, is easier to modify. And while it may look complicated, it is not, in reality that much more complicated than just writing a function.
Classes: Attributes and Methods
The first concept to understand is that of a class. A class is an abstract entity that contains atrtibutes and methods. We will see in the next section that objects are specific instantiations of a class. To solve the least squares problem we introduced in the previous section using object oriented programming, we define a class to store the matrix $\bbA$ and the vector $\bby$ and create a method in the class that solves the least squares problem:
class LinearFunction:
import numpy as np
class LeastSquares():
def __init__(self, A, y):
self.A = A
self.y = y
def solve(self):
A = self.A
y = self.y
AT = np.transpose(A) # Compute $\bbA^T$}
ATA = np.matmul(A,AT) # Compute $\bbA^T\bbA$}
iATA = np.linalg.inv(ATA) # Compute $(\bbA^T\bbA)^{-1}$}
iATAAT = np.matmul(AT,iATA) # Compute $(\bbA^T\bbA)^{-1}\bbA^T$}
x = np.matmul(y,iATAAT) # Compute $(\bbA^T\bbA)^{-1}\bbA^T\bby$}
return x
The class definition contains two methods. The method $\p{\_\_init\_\_}$ plays a special role in the creation of objects which we will explain soon. At this point, observe how it specifies the attributes that are part of the class. In this specific example, the class contains two attributes, the matrix $\p{A}$ and the vector $\p{y.}$ When we define a class, the $\p{\_\_init\_\_}$ function has to be specified always and $\p{self}$ has to always be the first parameter of the $\p{\_\_init\_\_}$ method. This is Python syntax.
The other function, $\p{solve,}$ is a function proper, which in object oriented programming we call a method. This method returns the solution of the least squares problem $\p{x}$ associated with the matrix $\p{A}$ and the vector $\p{y.}$ The matrix $\p{A}$ and the vector $\p{y}$ are not inputs to this function. They are attributes that belong to the class. Further notice that $\p{self}$ is the first parameter of the $\p{evaluate}$ method. Any method that is defined in a class has to take $\p{self}$ as the first parameter. This is just Python syntax as well.
Objects: Concrete Instances of Abstract Classes
The class is an abstract entity with methods that specify how to manipulate its attributes. If we want to actually process data, we create a specific instance. This is an object. For example, if we are interested in the same least squares problem we consider in the first section, namely, a random matrix and a random vector with Gaussian entries and dimensions $m=13$ and $n=3$, we create the following object as an instance of the class $\p{LeastSquares}$,
class LinearFunction: m = 13 n = 3 A = np.random.normal(0, 1, size=(n, m)) y = np.random.normal(0, 1, size=(1, m)) LSobject = LeastSquares(A, y)
When creating the object $\p{LSobject}$ we are implicitly calling the method $\p{LeastSquares.\_\_init\_\_.}$ In doing so we instantiate the attributes that belong to the object. If we now want to implement the solution of a least squares problem associated with this matrix and this vector we invoke the method $\p{solve}$ of the object $\p{LSobject}$
class LinearFunction: x = LSobject.solve()
The creation of an object and the invocation of a method are slightly more complicated than the single invocation of a function. But in the end the difference is minimal.
Whenever the object $\p{LSobject}$ is referenced in the code, we are referring to the least squares problem associated with the specific matrix $\p{A}$ and the specific vector $\p{y}$ that we passed during the creation of the object. If we wanted to have a different least squares problem, we could do so by instantiating another object of the $\p{LeastSquares}$ class,
class LinearFunction: AnotherA = np.random.normal(0, 1, size=(n, m)) Anothery = np.random.normal(0, 1, size=(1, m)) AnotherLSobject = ls.LeastSquares(AnotherA, Anothery)
If we now want to implement the solution of the least squares for this matrix-vector pair we invoke the $\p{solve}$ method of this specific object:
class LinearFunction: x = AnotherLSobject.solve()
The main difference between the use of classes, as we do in this section, and functions, as we did in the previous one, is one of frame of mind. When writing functions, the parameters that we pass are separate from the function. When writing a class, the parameters and the methods are integral parts of the object. Thinking in terms of objects helps our human brains in several ways. For example, having $\p{A}$ and $\p{y}$ encapsulated inside the object $\p{LSobject}$ while having $\p{AnotherA}$ and $\p{Anothery}$ encapsulated inside the object $\p{AnotherLSobject}$ reduces the likelihood that we mix up $\p{A}$ and $\p{y}$ with $\p{AnotherA}$ and $\p{Anothery}$. There are less things to remember. This advantage is minimal in this problem with two attributes and one method. However, it is not difficult to appreciate the advantage of this frame of mind when we have classes with large numbers of attributes and methods. This is particularly advantageous when we want to modify code an old piece of code or when we want to share code with other developers.
In any event, discussing the relative merits of using object oriented programming is outside of scope for us. The main reason for us to use it is that it is customary in data sciences and we will follow custom.
Inheritance
A third concept of object oriented programming we have to introduce is inheritance. This is the possibility of defining a “child” class that inherits methods from a “parent” class. As an example, suppose that we intend to create several random Gaussian least squares problems. Thus, instead of generating several matrices and vectors to pass as arguments in the creation of several different objects, it is more convenient to encapsulate the generation of the Gaussian matrix and vector inside of an object. To do that, create a class $\p{GaussianLeastSquares}$ which we define as a child of the $\p{LeastSquares}$ class,
class LinearFunction:
class GaussianLeastSquares(LeastSquares):
def __init__(self, m, n):
self.A = np.random.normal(0, 1, size=(n, m))
self.y = np.random.normal(0, 1, size=(n, m))
The specification of $the class \p{GaussianLeastSquares}$ as a child of the class $\p{LeastSquares}$ is done by making the latter an argument in the $\p{class}$ statement. The use of inheritance allows us to reuse our hard work in the creation of the $\p{LeastSquares}$ class. We do not need to specify the $\p{solve}$ function for the $\p{GaussianLeastSquares}$ because we are reusing from the parent class $\p{LeastSquares}$. We are inheriting, to use the more technical term. If at some point in the future we update the $\p{solve}$ method in the $\p{LeastSquares}$ class, that updated method is automatically inherited by the child class.
With this new class, the creation of least squares problem with a random matrix and vector simplifies to the code
class LinearFunction: m = 13 n = 3 LSobject = GaussianLeastSquares(m,n)
The code for the evaluation of the solution of the least squares problem is still the same because it has been inherited.
The most important advantage of defining a new class is that modifications to the class will now propagate to all the places where a Gaussian least squares problem is defined. If at some point in the future we decide that variance $2$ is more appropriate, it’s just a matter of changing the definition of the $\p{GaussianLeastSquares.\_\_init\_\_}$ method. The change will propagate to all the places where we instantiate an object belonging to the $\p{GaussianLeastSquares}$ class.
Code links
The code described here can be downloaded from the folder oop_python.zip. This folder contains the following three files:
$\p{least\_squares.py}$: The class $\p{LeastSquares}$ and the child class $\p{GaussianLeastSquares}$ are specified in this file. This is where the important code is written, but the file itself does not perform any computation. The other two files are the ones that are executable.
$\p{least\_squares\_main.py}$: This file instantiates an object of the class $\p{LeastSquares}$ and executes the $\p{solve}$ method. The matrix $\p{A}$ and the vectors $\p{y}$ and $\p{x}$ are printed.
$\p{gaussian\_least\_squares\_main.py}$: This files instantiates an object of the class $\p{GaussianLeastSquares}$ and executes the $\p{solve}$ method inherited from the parent class. We print the matrix $\p{A}$ and the vectors $\p{y}$ and $\p{x}$.