ForwardFinetuner

embetter.finetune.ForwardFinetuner

Create a feed forward model to finetune the embeddings towards a class.

Parameters

Name Type Description Default
hidden_dim The size of the hidden layer 50
n_epochs The number of epochs to run the optimiser for 500
learning_rate The learning rate of the feed forward model 0.01

fit(self, X, y)

Show source code in finetune/_forward.py
43
44
45
    def fit(self, X, y):
        """Fits the finetuner."""
        return self.partial_fit(X, y, classes=np.unique(y))

Fits the finetuner.

partial_fit(self, X, y, classes=None)

Show source code in finetune/_forward.py
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
    def partial_fit(self, X, y, classes=None):
        """Fits the finetuner using the partial_fit API."""
        if not hasattr(self, "_classes"):
            if classes is None:
                raise ValueError("`classes` must be provided for partial_fit")
            self._classes = classes
        # Create a model if it does not exist yet.
        if not hasattr(self, "_model"):
            self._model = FeedForwardModel(
                X.shape[1], self.hidden_dim, len(self._classes)
            )
            self._optimizer = torch.optim.Adam(
                self._model.parameters(), lr=self.learning_rate
            )
            self._criterion = nn.CrossEntropyLoss()

        torch_X = torch.from_numpy(X).detach().float()
        torch_y = torch.from_numpy(np.array(y)).detach()

        for _ in range(self.n_epochs):
            self._optimizer.zero_grad()
            out = self._model(torch_X)
            loss = self._criterion(out, torch_y)
            loss.backward()
            self._optimizer.step()

        return self

Fits the finetuner using the partial_fit API.

transform(self, X, y=None)

Show source code in finetune/_forward.py
75
76
77
78
    def transform(self, X, y=None):
        """Transforms the data according to the sklearn api by using the hidden layer."""
        Xt = torch.from_numpy(X).float().detach()
        return self._model.embed(Xt).detach().numpy()

Transforms the data according to the sklearn api by using the hidden layer.