Grep is unusably slow

[lynndylanhurley]

Description

Grep hangs for 10+ seconds when used in a large codebase.

Expected Behavior

I'm coming from FZF, which seems to stream the results in as they become available. The UI never hangs and I get instant feedback as I type.

Actual Behavior

Telescope seems to hang until all the results have been processed, which sometimes takes 10+ seconds in large codebases.

In this .gif I'm typing continuously, but as you can see the input is frozen for several seconds:

In this horrible codebase the only way to find things is to grep for them, so this is quite frustrating.

Details

• nvim version: NVIM v0.5.0-dev+9223d1450
• Operating system: MacOS 11.1
• Telescope commit: 5d121ee
• ripgrep 11.0.2

Configuration

" relevant plugins
Plug 'nvim-lua/popup.nvim'
Plug 'nvim-lua/plenary.nvim'
Plug 'romgrk/fzy-lua-native', { 'do': 'make' }
Plug 'nvim-telescope/telescope.nvim'
Plug 'nvim-telescope/telescope-fzf-writer.nvim'
Plug 'nvim-telescope/telescope-fzy-native.nvim'
Plug 'nvim-telescope/telescope-vimspector.nvim'

" telescope config
nnoremap <leader>t :lua require('telescope').extensions.fzf_writer.files()<CR>
nnoremap <leader>/ :lua require('telescope').extensions.fzf_writer.staged_grep()<CR>
nnoremap <leader>b <cmd>Telescope buffers<cr>
nnoremap <leader>c :lua require('telescope.builtin').git_bcommits()<cr>

lua <<EOF
require('telescope').setup {
    extensions = {
        fzf_writer = {
            use_highlighter = true
        }
    }
}
require('telescope').load_extension('fzy_native')
EOF

[kkharji]

@lynndylanhurley it would be nice if you have this issue open in fzf-telescope so that @tjdevries when god well, and he has time to refactor 😆 he will take notice of the issue.

but as far as I know about fzf_writer is:

• There is ongoing issue with performance when enabling the highlighter use_highlighter = true
• Increasing minimal characters to trigger the fzf does improve performance by a lot . eg. minimum_grep_characters = 3, minimum_files_characters = 3 in fzf_writer dict

[lynndylanhurley]

Thanks @tami5, I'll raise the issue in the fzf-telescope repo as well.

The problem also exists with the built-in live_grep function, i.e.:

nnoremap <leader>/ :lua require('telescope.builtin').live_grep()<CR>

I tried using fzf_writer with the hope that it would resolve the issue, but it only improved the hang time slightly.

I also tried setting use_highlighter = false and I didn't notice a difference.

Increasing the minimum_grep_characters does help slightly but it doesn't seem like a real solution. For example, when the results finally load in and I add or remove a character, the UI hangs again for several seconds with each key press. It's so bad that it's hard to keep focus on what I'm searching for.

[kkharji]

@lynndylanhurley I think this might fix our issue with current problems with live_grep and grep_string, #386 please test it out by doing gh pr checkout 386 in the repo root, or something similar.

[lynndylanhurley]

Thanks @tami5 ! I just pulled down and tested, but I'm still getting the same slowness. There does seem to be an additional UI update that happens before the final results render but the input is still frozen for several seconds:

[tssm]

For reference, if you want to replicate, I'm seeing the same while grepping the nixpkgs repo (which I do often) with the fzy-native extension enabled and disabled, which makes me thinks it is a UI issue as lynndylanhurley has pointed out

[Conni2461]

There is an async take 2 which comes closer to not blocking input #457. If i recall correctly it doesn't have on live_grep tho, because the string processing, still happens in main thread which makes it block.

Tj was talking about continuing this work in a take 3 if i recall correctly.

For now, what i am doing is this. Most of the time i already know what i am looking for, so i basically prefilter the results.

require'telescope.builtin'.grep_string{ only_sort_text = true, search = vim.fn.input("Grep For >") }

[dagadbm]

just curious to ask how this is moving along. I noticed the plenary issue was already merged and there are plans to make the async/await part of neovim core.
Any news?

[elianiva]

see #709 for progress

[rnevius]

Looks like #709 did not include making live_grep() async.

[clason]

No, as it says right in the commit message :)

live_grep repeatedly spawns an external job (rather than doing it once and then continuously) filtering the results and so requires nvim-lua/plenary.nvim#113.