a question

[unnir]

Hi,

first thank you for your repo!

I just want to ask you, is there a source from where I can understand more about the architecture of your work, besides the mentioned paper?
I mean, maybe you wrote a blog post about your algorithm or something like this.

Kind regards,
Paddy

update:

Also, could you please tell how I can reuse the model which I've trained. I mean I finished the training and want to apply it for sorting a list, f.e.:

python model.py [3,4,6,5,1,2]

And the output would be a sorted list:

[1,2,3,4,5,6]

[pemami4911]

Hey, thanks for the comments. I am currently in the process of getting results for experiments on the Travelling Salesman Problem, and in the process I've been improving the code quite a bit. As soon as I get some reasonable results that are comparable to those from the paper, I'll make a more detailed write-up (probably a blog post!). This was quite difficult to implement from scratch lol.

To do that, you'd just have to write a script that loads the model from a checkpoint (I have this code written in trainer.py, see line 112) and then takes the input string of numbers and passes it to the model, basically following the code I wrote in trainer.py for evaluating the model from line 250 onwards. Line 285 is an example of how to get the final result out.

[unnir]

thank you for your answer and looking 👀 forward to your blog post!

[ricgama]

Hello,
Very nice work. I'm also working on an implementation of the paper's solution to solve TSP in Pytorch. I've also implemented the Supervised learning version of Vinyals et al.
I've a question regarding your code: Why have you implemented an LSTM cell in the decoder, insted of using the Pytorch LSTM implementation?

Would you be interested in exchanging implementation ideas? If so I can send you an email...

Best regards,
Ricardo

[unnir]

I'm kindly asking to keep the discussion here, because it will be beneficial for all concerned.

[pemami4911]

@ricgama I implemented the LSTM cell in the decoder myself to add the "glimpse" and "pointer" attention components manually. I took inspiration from https://github.com/MaximumEntropy/Seq2Seq-PyTorch/blob/master/model.py

[ricgama]

ok, thanks for the reply. As I followed https://github.com/spro/practical-pytorch/tree/master/seq2seq-translation I've used an Pytorch cell for the decoder as well. I think it's the same only different programming style.

[pemami4911]

I'm not 100% sure, but I don't think it is the same. The attending and pointing mechanisms, as indicated in the paper, take as input both the encoder LSTM's hidden state (context, in the NCO for RL paper, ref) as well as the decoder hidden state at each time step of the decoder LSTM (the query). It looks like in that repo that the attention is only over the encoder hidden states. No clue as to how this would empirically affect performance without trying it.

[ricgama]

My decoder class looks like this:

class Decoder(nn.Module):
    def __init__(self, feactures_dim,hidden_size, n_layers=1):
        super(Decoder, self).__init__()
                
        self.W1 = Var(hidden_size, hidden_size)
        self.W2 = Var(hidden_size, hidden_size)
        self.b2 = Var(hidden_size)
        self.V = Var(hidden_size)

        self.W1g = Var(hidden_size, hidden_size)
        self.W2g = Var(hidden_size, hidden_size)
        self.b2g = Var(hidden_size)
        self.Vg = Var(hidden_size)
        
        self.gru = nn.GRU(hidden_size, hidden_size, batch_first=True, num_layers=n_layers)

    def pointerAtt(self,di, enc_outputs):
        w1e = torch.matmul(enc_outputs,self.W1)
        w2h = (torch.matmul(di, self.W2) + self.b2).unsqueeze(1)
        u = F.tanh(w1e + w2h)
        a = torch.matmul(u, self.V)
        a = F.log_softmax(a)
        return a
    
    def glimpseAtt(self,di, enc_outputs):
        w1eg = torch.matmul(enc_outputs,self.W1g)
        w2hg = (torch.matmul(di, self.W2g) + self.b2g).unsqueeze(1)        
        ug = F.tanh(w1eg + w2hg)
        a = torch.matmul(ug, self.Vg)
        a = F.softmax(a).unsqueeze(1)
        att_state = torch.bmm(a,enc_outputs).squeeze(1)
        return att_state
    
    def forward(self, input, hidden, enc_outputs):
        di = hidden[-1]
        h = self.glimpseAtt(di, enc_outputs)
        a = self.pointerAtt(h, enc_outputs)
        res, hidden = self.gru(input, hidden)  
        return a, hidden

Maybe I'm wrong but I think that it does what you have described. What do you think?

[pemami4911]

I see, you're calling forward repeatedly and passing the last decoder hidden state as input. Yeah that should be equivalent!

[pemami4911]

@ricgama How is it going for you? I have yet to get results on TSP20. The hyperparameters mentioned in the paper don't seem to be working for me :(

[ricgama]

Hello @pemami4911,
I’m still fighting with it. I’m testing first for n=10 as is an easier problem and I can obtain very good results with the supervised learning version. For the RL, the critic loss decreases very well but I guess that I still have a bug in my code because the tour distance stays practically the same…
Have you manage to train for a small n? For how many epochs?
Although I understand @unnir argument, I still think that via email we could improve our discussions about this, even exchanging emails between the 3. My email is rgama@estgl.ipv.pt.
Best regards,
Ricardo

[pemami4911]

@ricgama Is there a particular difficulty with using Github issues for you? I find communicating via Issues quite nice actually.

Yeah for n=10, I tried halving the size of the network to 64 and batch sizes of 64, and for learning rates of ~3.5e-4 I was able to get a little bit of improvement. But for n=20, and for the most part for n=10, the tour length stays the same. I'm plotting the average negative-log likelihood of the selected tours, which helps to see whether it's learning something our not. On n=10, the NLL hangs at 15.10; on my best run, it dropped all the way down to ~5 for a little while, but the test tour length stayed around 5. I've been training the regular 128 dim network with lr 1e-3 on n=20 for about ~30 hours now (~850K minibatches of size 128) and the NLL has stayed flat and the tour length hasn't budged :/

[pemami4911]

@ricgama are you setting the logits of the already selected nodes during decoding to -infty to ensure only valid tours are used?

[pemami4911]

@ricgama Also, did you do a distributed implementation? Or single-threaded on a GPU?

[ricgama]

@pemami4911
1- Yes, I'm setting the logits to -inf
2- I have a single-thead on a GPU
I don't have any values for now, because I'm still trying to find the problem in the code.
Here it is my training plot for superfised version and n=10. At the end of 10 epochs I obtain:
true: 3.06973212456 predicted: 3.13004359636
on the Vinyals "Pointer Networks" paper test set.