Skip to content

Commit fb2dfc5

Browse files
committed
bpf: arena: make arena kfuncs any context safe
Make arena related kfuncs any context safe by the following changes: bpf_arena_alloc_pages() and bpf_arena_reserve_pages(): Replace the usage of the mutex with a rqspinlock for range tree and use kmalloc_nolock() wherever needed. Use free_pages_nolock() to free pages from any context. apply_range_set/clear_cb() with apply_to_page_range() has already made populating the vm_area in bpf_arena_alloc_pages() any context safe. bpf_arena_free_pages(): defer the main logic to a workqueue if it is called from a non-sleepable context. specialize_kfunc() is used to replace the sleepable arena_free_pages() with bpf_arena_free_pages_non_sleepable() when the verifier detects the call is from a non-sleepable context. In the non-sleepable case, arena_free_pages() queues the address and the page count to be freed to a lock-less list of struct arena_free_spans and raises an irq_work. The irq_work handler calls schedules_work() as it is safe to be called from irq context. arena_free_worker() (the work queue handler) iterates these spans and clears ptes, flushes tlb, zaps pages, and calls __free_page(). Signed-off-by: Puranjay Mohan <[email protected]>
1 parent 4c4fa18 commit fb2dfc5

File tree

3 files changed

+203
-40
lines changed

3 files changed

+203
-40
lines changed

include/linux/bpf.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -673,6 +673,8 @@ void bpf_map_free_internal_structs(struct bpf_map *map, void *obj);
673673
int bpf_dynptr_from_file_sleepable(struct file *file, u32 flags,
674674
struct bpf_dynptr *ptr__uninit);
675675

676+
void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt);
677+
676678
extern const struct bpf_map_ops bpf_map_offload_ops;
677679

678680
/* bpf_type_flag contains a set of flags that are applicable to the values of

0 commit comments

Comments
 (0)