[BUG] "column id not found in data" error when binding

[sadettindemirel]

Please confirm the following

• I have searched the existing issues
• The behaviour of the program is deviated from what is described in the documentation.
• I can reproduce this problem for more than one time.
• This is NOT a 3-digit error -- it does not display an error message like something went wrong. Status code: 400.
• This is a 3-digit error and I have consulted the Understanding API errors vignette and the suggestions do not help.

Describe the bug

I'm having problem with bind_tweets function. When I tried to bind the json files into tidy format, error below happened.

my code to get tweets from Twitter API

get_all_tweets(
    users = c("Reuters"),
    start_tweets = "2020-01-01T00:00:00Z",
    end_tweets = "2021-01-01T00:00:00Z",
    data_path = "reuterstweets",
    n= Inf,
    is_retweet = FALSE,
    bind_tweets = FALSE
  )

Binding files

r_tweets <- bind_tweets(data_path = "reuterstweets", output_format = "tidy")

The Error

rlang::last_error()
<error/rlang_error_data_pronoun_not_found>
Column `id` not found in `.data`
Backtrace:
  1. academictwitteR::bind_tweets(data_path = "reuterstweets", output_format = "tidy")
 29. rlang:::abort_data_pronoun(x)
Run `rlang::last_trace()` to see the full context.

Detailed error message

> rlang::last_trace()
<error/rlang_error_data_pronoun_not_found>
Column `id` not found in `.data`
Backtrace:
     x
  1. +-academictwitteR::bind_tweets(data_path = "reuterstweets", output_format = "tidy")
  2. | \-academictwitteR:::.flat(data_path, output_format = output_format)
  3. |   \-purrr::map_dfr(data_files, convert_json, output_format = output_format)
  4. |     \-purrr::map(.x, .f, ...)
  5. |       \-academictwitteR:::.f(.x[[i]], ...)
  6. |         +-`%>%`(...)
  7. |         +-dplyr::select(...)
  8. |         \-dplyr:::select.data.frame(...)
  9. |           \-tidyselect::eval_select(expr(c(...)), .data)
 10. |             \-tidyselect:::eval_select_impl(...)
 11. |               +-tidyselect:::with_subscript_errors(...)
 12. |               | +-base::tryCatch(...)
 13. |               | | \-base:::tryCatchList(expr, classes, parentenv, handlers)
 14. |               | |   \-base:::tryCatchOne(expr, names, parentenv, handlers[[1L]])
 15. |               | |     \-base:::doTryCatch(return(expr), name, parentenv, handler)
 16. |               | \-tidyselect:::instrument_base_errors(expr)
 17. |               |   \-base::withCallingHandlers(...)
 18. |               \-tidyselect:::vars_select_eval(...)
 19. |                 \-tidyselect:::walk_data_tree(expr, data_mask, context_mask)
 20. |                   \-tidyselect:::eval_c(expr, data_mask, context_mask)
 21. |                     \-tidyselect:::reduce_sels(node, data_mask, context_mask, init = init)
 22. |                       \-tidyselect:::walk_data_tree(new, data_mask, context_mask)
 23. |                         \-base::eval(expr, data_mask)
 24. |                           \-base::eval(expr, data_mask)
 25. |                             +-id
 26. |                             \-rlang:::`$.rlang_data_pronoun`(.data, id)
 27. |                               \-rlang:::data_pronoun_get(x, nm)
 28. +-dplyr::distinct(., .data$id, .keep_all = TRUE)
 29. \-rlang:::abort_data_pronoun(x)

Expected Behavior

Normally I would expect bind_tweets command to bind json files and create a tidy format

Steps To Reproduce

reprex::reprex()
#> x Install the styler package in order to use style = TRUE.
#> i Non-interactive session, setting html_preview = FALSE.
#> CLIPR_ALLOW has not been set, so clipr will not run interactively
#> Error in switch(where, expr = stringify_expression(x_expr), clipboard = ingest_clipboard(), : EXPR must be a length 1 vector

Environment

sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] dplyr_1.0.7 lubridate_1.8.0 here_1.0.1
[4] jsonlite_1.7.2 academictwitteR_0.3.0

loaded via a namespace (and not attached):
[1] compiler_4.1.2 pillar_1.6.4 tools_4.1.2 extrafont_0.17
[5] digest_0.6.28 evaluate_0.14 lifecycle_1.0.1 tibble_3.1.6
[9] pkgconfig_2.0.3 rlang_0.4.12 rstudioapi_0.13 DBI_1.1.1
[13] curl_4.3.2 yaml_2.2.1 xfun_0.28 fastmap_1.1.0
[17] Rttf2pt1_1.3.9 httr_1.4.2 knitr_1.36 generics_0.1.1
[21] fs_1.5.0 vctrs_0.3.8 rprojroot_2.0.2 tidyselect_1.1.1
[25] glue_1.5.0 R6_2.5.1 fansi_0.5.0 rmarkdown_2.11
[29] tidyr_1.1.4 purrr_0.3.4 extrafontdb_1.0 magrittr_2.0.1
[33] htmltools_0.5.2 ellipsis_0.3.2 usethis_2.1.3 assertthat_0.2.1
[37] utf8_1.2.2 crayon_1.4.2

Anything else?

I didn't get this error before. Thanks in advance

[chainsawriot]

@sadettindemirel

Thanks for reporting this. I can confirm with you that I can reproduce the bug. Sorry for the bug. And I can also pinpoint which json file and which line of code is causing the problem.

academictwitteR/R/bind_tweets.R

Line 119 in 85f82a8

source_main <- dplyr::select(raw$sourcetweet.main, .data$id, .data$text, .data$lang, .data$author_id) %>%

A hot fix is developing.

[chainsawriot]

@sadettindemirel Fixed. Please give it a try.

Once again, sorry for the bug, and thanks a lot for reporting this.

[sadettindemirel]

@chainsawriot it works now. Great work, Many thanks.