Logistic Regression - numpy.float64

Question

I am trying to create Logistic Regression from scratch using the Squared Loss function. However, I am having this error that I can't really figure out. The error is: can't multiply sequence by non-int of type 'numpy.float64'. Any help will be much appreciated! (yes I know I got lazy with the coefficient.)

def logisticReg(data):
  X_train = [(d[0],d[1],d[2]) for d, _ in data]
  Y_train = [y for _, y in data]
  
  LogReg = LogisticRegression(random_state=42, solver='sag', penalty='none', max_iter=10000, fit_intercept=False)
  LogReg.fit(X_train, Y_train)
  w=[round(c,2) for c in LogReg.coef_[0]]

  sigmoid = lambda y: 1/(1+np.exp(-y))
  classify = lambda y: 1 if y > 0.5 else 0
  F = lambda W, X: sum([w*x for w,x in zip(W,X)])

  for i in range(len(X_train)):
    Function = F(w,X_train)
    y_pred = sigmoid(Fucntion)
    Data_m = (-2) * sum(x*(y-y_pred)) 
    Data_b = (-2) * sum(y - y_pred)
    m = m- L*Data_m #update weights
    b = b-L*Data_b
  
  weights = zip(m,b)
  print(weights)

   
data = [((1, 0, 0), 1), ((1, 1, 7), 0), ((1, -3, -2), 0), ((1, 8, 9), 1), ((1, 4, 3), 1), ((1, 5, -2), 1), ((1, 0, 0), 1), ((1, 6, 9), 1), ((1, 4, 2), 1), ((1, 1, -9), 1), ((1, -7, 7), 0), ((1, 0, -1), 1), ((1, 9, -4), 1), ((1, 1, 0), 1), ((1, -2, -5), 1), ((1, 2, 3), 1), ((1, -7, 2), 0), ((1, -3, 0), 0), ((1, 5, 0), 1), ((1, 0, -3), 1), ((1, -2, 3), 0), ((1, 9, 6), 1), ((1, 0, -8), 1), ((1, 0, 2), 0), ((1, -8, 6), 0), ((1, 1, 9), 0), ((1, 0, 5), 0), ((1, -4, 9), 0), ((1, 8, 2), 1), ((1, 2, 6), 0)] 
logisticReg(data)

Answer 1

I have found out that the error occures on the call of function called F.

I fixed the problem by slightly modifying your code.

import numpy as np
import sklearn.linear_model as sk

def logisticReg(data):
    X_train = [(d[0], d[1], d[2]) for d, _ in data]
    Y_train = [y for _, y in data]
  
    LogReg = sk.LogisticRegression(random_state=42, solver='sag', penalty='l2', max_iter=10000, fit_intercept=False)
    LogReg.fit(X_train, Y_train)
    w=[round(c,2) for c in LogReg.coef_[0]]

    sigmoid = lambda y: 1/(1+np.exp(-y))
    thresh = lambda y: 1 if y > 0.5 else 0
    F = lambda W, X: sum([w*x for w,x in zip(np.array(W), np.array(X))])

    for i in range(len(X_train)):
          res = F(w, X_train)
          y_pred = sigmoid(res)
#    Data_m = (-2) * sum(x*(y - y_pred))
#    Data_b = (-2) * sum(y - y_pred)
#    m = m - L*Data_m
#    b = b - L*Data_b
  
#  weights = zip(m,b)
#  print(weights)
    
    return(None)
    
    
data = [((1, 0, 0), 1), ((1, 1, 7), 0), ((1, -3, -2), 0), ((1, 8, 9), 1), ((1, 4, 3), 1), ((1, 5, -2), 1), ((1, 0, 0), 1), ((1, 6, 9), 1), ((1, 4, 2), 1), ((1, 1, -9), 1), ((1, -7, 7), 0), ((1, 0, -1), 1), ((1, 9, -4), 1), ((1, 1, 0), 1), ((1, -2, -5), 1), ((1, 2, 3), 1), ((1, -7, 2), 0), ((1, -3, 0), 0), ((1, 5, 0), 1), ((1, 0, -3), 1), ((1, -2, 3), 0), ((1, 9, 6), 1), ((1, 0, -8), 1), ((1, 0, 2), 0), ((1, -8, 6), 0), ((1, 1, 9), 0), ((1, 0, 5), 0), ((1, -4, 9), 0), ((1, 8, 2), 1), ((1, 2, 6), 0)] 
logisticReg(data)

However, you should rethink about what your algorithm should do because it seems unclear: you want to code the Logistic Regression algorithm from scratch but use the algorithm from sklearn to compute the value of the coefficients of the model for a given dataset and then you seem to use these coefficients in the for loop...

What you should do in your Logistic Regression function is compute the coefficients of the Logistic Regression model applied on a given dataset without using sklearn at all. And then you may call your own Logistic Regression on a given dataset to get the coefficients and compare them to those computed by the Logistic Regression from sklearn.

Useful link to documentation about logistic regression model fitting: https://en.wikipedia.org/wiki/Logistic_regression#Model_fitting