Machine Learning Weight Optimization

Classes for defining neural network weight optimization problems.

class NeuralNetwork(hidden_nodes=None, activation='relu', algorithm='random_hill_climb', max_iters=100, bias=True, is_classifier=True, learning_rate=0.1, early_stopping=False, clip_max=10000000000.0, restarts=0, schedule=<mlrose.decay.GeomDecay object>, pop_size=200, mutation_prob=0.1, max_attempts=10, random_state=None, curve=False)

Class for defining neural network classifier weights optimization problem.

Parameters:

• hidden_nodes (list of ints) – List giving the number of nodes in each hidden layer.
• activation (string, default: ‘relu’) – Activation function for each of the hidden layers. Must be one of: ‘identity’, ‘relu’, ‘sigmoid’ or ‘tanh’.
• algorithm (string, default: ‘random_hill_climb’) – Algorithm used to find optimal network weights. Must be one of:’random_hill_climb’, ‘simulated_annealing’, ‘genetic_alg’ or ‘gradient_descent’.
• max_iters (int, default: 100) – Maximum number of iterations used to fit the weights.
• bias (bool, default: True) – Whether to include a bias term.
• is_classifer (bool, default: True) – Whether the network is for classification or regression. Set True for classification and False for regression.
• learning_rate (float, default: 0.1) – Learning rate for gradient descent or step size for randomized optimization algorithms.
• early_stopping (bool, default: False) – Whether to terminate algorithm early if the loss is not improving. If True, then stop after max_attempts iters with no improvement.
• clip_max (float, default: 1e+10) – Used to limit weights to the range [-1*clip_max, clip_max].
• restarts (int, default: 0) – Number of random restarts. Only required if algorithm = 'random_hill_climb'.
• schedule (schedule object, default = mlrose.GeomDecay()) – Schedule used to determine the value of the temperature parameter. Only required if algorithm = 'simulated_annealing'.
• pop_size (int, default: 200) – Size of population. Only required if algorithm = 'genetic_alg'.
• mutation_prob (float, default: 0.1) – Probability of a mutation at each element of the state vector during reproduction, expressed as a value between 0 and 1. Only required if algorithm = 'genetic_alg'.
• max_attempts (int, default: 10) – Maximum number of attempts to find a better state. Only required if early_stopping = True.
• random_state (int, default: None) – If random_state is a positive integer, random_state is the seed used by np.random.seed(); otherwise, the random seed is not set.
• curve (bool, default: False) – If bool is True, fitness_curve containing the fitness at each training iteration is returned.

Variables:

• fitted_weights (array) – Numpy array giving the fitted weights when fit is performed.
• loss (float) – Value of loss function for fitted weights when fit is performed.
• predicted_probs (array) – Numpy array giving the predicted probabilities for each class when predict is performed for multi-class classification data; or the predicted probability for class 1 when predict is performed for binary classification data.
• fitness_curve (array) – Numpy array giving the fitness at each training iteration.

class LinearRegression(algorithm='random_hill_climb', max_iters=100, bias=True, learning_rate=0.1, early_stopping=False, clip_max=10000000000.0, restarts=0, schedule=<mlrose.decay.GeomDecay object>, pop_size=200, mutation_prob=0.1, max_attempts=10, random_state=None, curve=False)

Class for defining linear regression weights optimization problem. Inherits fit and predict methods from NeuralNetwork() class.

Parameters:

• algorithm (string, default: ‘random_hill_climb’) – Algorithm used to find optimal network weights. Must be one of:’random_hill_climb’, ‘simulated_annealing’, ‘genetic_alg’ or ‘gradient_descent’.
• max_iters (int, default: 100) – Maximum number of iterations used to fit the weights.
• bias (bool, default: True) – Whether to include a bias term.
• learning_rate (float, default: 0.1) – Learning rate for gradient descent or step size for randomized optimization algorithms.
• early_stopping (bool, default: False) – Whether to terminate algorithm early if the loss is not improving. If True, then stop after max_attempts iters with no improvement.
• clip_max (float, default: 1e+10) – Used to limit weights to the range [-1*clip_max, clip_max].
• restarts (int, default: 0) – Number of random restarts. Only required if algorithm = 'random_hill_climb'.
• schedule (schedule object, default = mlrose.GeomDecay()) – Schedule used to determine the value of the temperature parameter. Only required if algorithm = 'simulated_annealing'.
• pop_size (int, default: 200) – Size of population. Only required if algorithm = 'genetic_alg'.
• mutation_prob (float, default: 0.1) – Probability of a mutation at each element of the state vector during reproduction, expressed as a value between 0 and 1. Only required if algorithm = 'genetic_alg'.
• max_attempts (int, default: 10) – Maximum number of attempts to find a better state. Only required if early_stopping = True.
• random_state (int, default: None) – If random_state is a positive integer, random_state is the seed used by np.random.seed(); otherwise, the random seed is not set.
• curve (bool, default: False) – If bool is true, curve containing the fitness at each training iteration is returned.

Variables:

• fitted_weights (array) – Numpy array giving the fitted weights when fit is performed.
• loss (float) – Value of loss function for fitted weights when fit is performed.
• fitness_curve (array) – Numpy array giving the fitness at each training iteration.

class LogisticRegression(algorithm='random_hill_climb', max_iters=100, bias=True, learning_rate=0.1, early_stopping=False, clip_max=10000000000.0, restarts=0, schedule=<mlrose.decay.GeomDecay object>, pop_size=200, mutation_prob=0.1, max_attempts=10, random_state=None, curve=False)

Class for defining logistic regression weights optimization problem. Inherits fit and predict methods from NeuralNetwork() class.

Parameters:

• algorithm (string, default: ‘random_hill_climb’) – Algorithm used to find optimal network weights. Must be one of:’random_hill_climb’, ‘simulated_annealing’, ‘genetic_alg’ or ‘gradient_descent’.
• max_iters (int, default: 100) – Maximum number of iterations used to fit the weights.
• bias (bool, default: True) – Whether to include a bias term.
• learning_rate (float, default: 0.1) – Learning rate for gradient descent or step size for randomized optimization algorithms.
• early_stopping (bool, default: False) – Whether to terminate algorithm early if the loss is not improving. If True, then stop after max_attempts iters with no improvement.
• clip_max (float, default: 1e+10) – Used to limit weights to the range [-1*clip_max, clip_max].
• restarts (int, default: 0) – Number of random restarts. Only required if algorithm = 'random_hill_climb'.
• schedule (schedule object, default = mlrose.GeomDecay()) – Schedule used to determine the value of the temperature parameter. Only required if algorithm = 'simulated_annealing'.
• pop_size (int, default: 200) – Size of population. Only required if algorithm = 'genetic_alg'.
• mutation_prob (float, default: 0.1) – Probability of a mutation at each element of the state vector during reproduction, expressed as a value between 0 and 1. Only required if algorithm = 'genetic_alg'.
• max_attempts (int, default: 10) – Maximum number of attempts to find a better state. Only required if early_stopping = True.
• random_state (int, default: None) – If random_state is a positive integer, random_state is the seed used by np.random.seed(); otherwise, the random seed is not set.
• curve (bool, default: False) – If bool is true, curve containing the fitness at each training iteration is returned.

Variables:

• fitted_weights (array) – Numpy array giving the fitted weights when fit is performed.
• loss (float) – Value of loss function for fitted weights when fit is performed.
• fitness_curve (array) – Numpy array giving the fitness at each training iteration.