Skip to content

Conversation

MrYxJ
Copy link

@MrYxJ MrYxJ commented Mar 18, 2024

Fixed a minor encoding error in the mask matrix size in one_file_ref.py where the original mask would display one more column than expected size.

fix sliding window mask size bug
@MrYxJ
Copy link
Author

MrYxJ commented Mar 18, 2024

tensor = torch.ones((5, 5))
sliding_window = 3

mask = torch.tril(tensor, diagonal=0)
# make the mask banded to account for sliding window
# mask = torch.triu(mask, diagonal=-sliding_window)  # actually produces a large column of slide window size
mask = torch.triu(mask, diagonal=-sliding_window+1)  
mask = torch.log(mask)
mask
tensor([[0., -inf, -inf, -inf, -inf],
        [0., 0., -inf, -inf, -inf],
        [0., 0., 0., -inf, -inf],
        [-inf, 0., 0., 0., -inf],
        [-inf, -inf, 0., 0., 0.]])

@MrYxJ MrYxJ changed the title Update one_file_ref.py Fix sliding window mask size in one_file_ref.py Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant