Machine learning with python

Correlation matrix#

The np corrcoef calculates the correlation matrix.

np.random.normal(loc=0.0, scale=1.0) # loc (mean), scale (standard deviation)
np.corrcoef(x, y)

# Generates a random number with seed 1303, reproduci
rng = np.random.default_rng(1303)
rng.normal(scale=10, size=5)

rng.normal and rng.standard_normal.

Sequences#

# Creates a linearly spaced array of 11 elements, starting at 0 to 10.
seq1 = np.linspace(0, 10, 11)

# Similar, but an array is obtained by specifying the step size.
seq2 = np.arange(0, 10, step=1)

The slice notation can be used to retrieve a subset of an array from the i-th to j-th index.

seq[0:3]
seq.slice(3, 6)

Lets create a 4x4 matrix. We can select rows and columns using the : notation.

A = np.arrange(16).reshape((4,4))

# Select the first two rows
A[[0,1]]

# Select the third and fourth column.
A[:,[2,3]]

A submatrix can be constructed using:

idx = np.ix_([2,3],[2,3])
A[idx]

Pandas#

A data frame represents tabular data, where each row or column may be given names. CSV files are an example of such data. Pandas is best suited to deal with this data.

import pandas as pd
data = pd.read_csv('dataset.csv')
data = pd.read_csv('dataset.csv', na_values=['-'])

data['age'] # selects the age column
data.shape  # describes the dimensions of the data.

# To remove not available data.
data.dropna()

data.columns    # describes the column names
data[:3]        # selects the first three rows of the data frame

You can select a subset of rows based on criteria as follows. Note that it's using a boolean array to select the individual rows

age_over_40 = data['age'] > 40
data[age_over_40]

Selecting columns can be done by passing in a list of column names:

data[['age', 'height']]

By default each row has a unique index number. You can use one of the columns to define the name for each of the observations. Then it is also possible to select rows based on name.

data = data.set_index('name')

persons = ['Ruud', 'Katarina']
data.loc[person]
data.iloc[[10,12]]

You could therefore select a subset of the data frame using: data.loc[lambda df: df['age'] > 40, ['age', 'height']].

Zip#

You can iterate element-wise over two arrays using zip. Per iteration it returns a tuple representing the i-th element of each of the arrays we are zipping over.