Introduction to Deep Learning¶
We will start with a simple example to introduce Deep Learning
- Input (Test & Train)
- Output
- Model - Architecture & Layers & Weights & Activation
- Loss Function
- Optimization
Learn a Noisy Function¶
Saddle Function => $$ Z = 2*X*X - 3*Y*Y + 5 + e$$
In [7]:
import numpy as np
import pandas as pd
import keras
import matplotlib.pyplot as plt # Visualisation Library
In [8]:
%matplotlib inline
In [9]:
x = np.arange(-1, 1, 0.01)
y = np.arange(-1, 1, 0.01)
In [11]:
X,Y = np.meshgrid(x,y)
c = np.ones([200, 200])
e = np.random.rand(200, 200)*0.1
In [12]:
Z = 2 *X*X - 3*Y*Y + 5*c + e
In [13]:
import sys
sys.path.append("../")
In [20]:
from mpl_toolkits.mplot3d import Axes3D
In [21]:
def plot3d(X,Y,Z):
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(111, projection='3d')
ax.plot_surface(X, Y, Z, color='y')
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
plt.show()
In [23]:
plot3d(X, Y, Z)
- Input => X, Y
- Output => Z
Using Keras to learn a "Non-Linear" Function¶
Step 0: Load the Keras Model¶
In [24]:
from keras.models import Sequential
from keras.layers import Dense
Step 1: Create the input and output data¶
In [30]:
input_xy = np.c_[X.reshape(-1), Y.reshape(-1)] ### X input in ML
output_z = Z.reshape(-1) ### y output in ML
In [31]:
input_xy.shape, output_z.shape
Out[31]:
Step 2: Create the Model (Tranformation + Regression)¶
In [132]:
model = Sequential()
model.add(Dense(4, input_dim=2, activation="linear"))
model.add(Dense(2, input_dim=2, activation="linear"))
model.add(Dense(1))
In [133]:
model.summary()
In [134]:
from keras.utils import plot_model
In [135]:
plot_model(model, show_layer_names=True, show_shapes=True)
Out[135]:
Step 3: Compile the Model: Loss, Optimizer, Fit the Model¶
In [136]:
model.compile(loss="mean_squared_error", optimizer="sgd", metrics=["mse"])
In [137]:
%%time
output = model.fit(input_xy, output_z, epochs=10, validation_split=0.2, shuffle=True, verbose=1)
In [138]:
output_df = pd.DataFrame(output.history)
output_df.head()
Out[138]:
In [139]:
output_df.plot.line(y=["val_loss", "loss"]);
Step 5:: Make prediction from model¶
In [140]:
Z_pred = model.predict(input_xy).reshape(200,200)
In [141]:
plot3d(X,Y, Z_pred)
Experiments¶
- Change the number of layers
- Change the number of learning units in a layer
- Change the activation from "relu" to "linear"
In [126]:
Dense?
In [ ]: