Re: [PATCH] Implement fast verisons of json-parse functions

[Eli Zaretskii]

> From: Herman, Géza <geza.herman@gmail.com>
> Cc: Géza Herman <geza.herman@gmail.com>,
>  emacs-devel@gnu.org
> Date: Sat, 23 Mar 2024 11:40:41 +0100
> 
> I attached the latest version of the parser.  I reverted the Lisp 
> vector based workspace to use a malloc based one, and added a note 
> regarding that the GC most not run during parsing.  I also 
> implemented some additional optimizations, this version runs ~10x 
> faster on my LSP dataset.  I also run the parser on all the JSON 
> files I found on my computer (~60,000 files, 1.5 GiB), no problems 
> were found.  The parser runs ~6.5x faster (3.5 sec vs 23.4 sec) on 
> this dataset.

Thanks.  I installed this on the master branch, after adding the
required commit log messages and some cleanup of unused functions.

However:

  . 3 tests in test/src/json-tests.el are now failing, where they
    succeeded before; see the log at the end
  . the times of the relevant tests don't seem to be faster than
    the libjansson version, perhaps because this is an unoptimized
    build

Here are the failed tests; please see how to fix them:

  Test json-parse-string/invalid-unicode backtrace:
    signal(ert-test-failed (((should-error (json-parse-string "[\"\200\"
    ert-fail(((should-error (json-parse-string "[\"\200\"]") :type 'json
    ert--should-error-handle-error((closure ((form-description-404 (shou
    (condition-case -condition- (unwind-protect (setq value-402 (apply f
    (let ((errorp405 nil) (form-description-fn-406 #'(lambda nil form-de
    (let (form-description-404) (let ((errorp405 nil) (form-description-
    (let ((value-402 'ert-form-evaluation-aborted-403)) (let (form-descr
    (let* ((fn-400 #'json-parse-string) (args-401 (condition-case err (l
    (closure (t) nil (let* ((fn-395 #'fboundp) (args-396 (condition-case
    #f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
    handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
    ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
    ert-run-test(#s(ert-test :name json-parse-string/invalid-unicode :do
    ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
    ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
    ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
    ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
    eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
    command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
    command-line()
    normal-top-level()
  Test json-parse-string/invalid-unicode condition:
      (ert-test-failed
       ((should-error (json-parse-string "[\"\200\"]") :type
                      'json-parse-error)
        :form (json-parse-string "[\"\200\"]") :condition
        (json-utf8-decode-error 1 3 3) :fail-reason
        "the error signaled did not have the expected type"))
     FAILED   7/21  json-parse-string/invalid-unicode (0.000198 sec) at 
src/json-tests.el:196
  Test json-parse-string/null backtrace:
    signal(ert-test-failed (((should-error (json-parse-string "\0") :typ
    ert-fail(((should-error (json-parse-string "\0") :type 'wrong-type-a
    ert--should-error-handle-error((closure ((form-description-377 (shou
    (condition-case -condition- (unwind-protect (setq value-375 (apply f
    (let ((errorp378 nil) (form-description-fn-379 #'(lambda nil form-de
    (let (form-description-377) (let ((errorp378 nil) (form-description-
    (let ((value-375 'ert-form-evaluation-aborted-376)) (let (form-descr
    (let* ((fn-373 #'json-parse-string) (args-374 (condition-case err (l
    (closure (t) nil (let* ((fn-368 #'fboundp) (args-369 (condition-case
    #f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
    handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
    ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
    ert-run-test(#s(ert-test :name json-parse-string/null :documentation
    ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
    ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
    ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
    ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
    eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
    command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
    command-line()
    normal-top-level()
  Test json-parse-string/null condition:
      (ert-test-failed
       ((should-error (json-parse-string "\0") :type 'wrong-type-argument)
        :form (json-parse-string "\0") :condition (json-end-of-file 1 1 1)
        :fail-reason "the error signaled did not have the expected type"))
     FAILED   8/21  json-parse-string/null (0.000281 sec) at 
src/json-tests.el:187
  Test json-parse-string/object backtrace:
    signal(ert-test-failed (((should (equal (json-parse-string input :ob
    ert-fail(((should (equal (json-parse-string input :object-type 'alis
    (if (unwind-protect (setq value-231 (apply fn-229 args-230)) (setq f
    (let (form-description-233) (if (unwind-protect (setq value-231 (app
    (let ((value-231 'ert-form-evaluation-aborted-232)) (let (form-descr
    (let* ((fn-229 #'equal) (args-230 (condition-case err (list (json-pa
    (let ((input "{ \"abc\" : [1, 2, true], \"def\" : null, \"abc\" : [9
    (closure (t) nil (let* ((fn-209 #'fboundp) (args-210 (condition-case
    #f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
    handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
    ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
    ert-run-test(#s(ert-test :name json-parse-string/object :documentati
    ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
    ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
    ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
    ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
    eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
    command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
    command-line()
    normal-top-level()
  Test json-parse-string/object condition:
      (ert-test-failed
       ((should
         (equal (json-parse-string input :object-type ...) '(... ...)))
        :form
        (equal ((abc . [1 2 t]) (def . :null) (abc . [9 :false]))
               ((abc . [9 :false]) (def . :null)))
        :value nil :explanation
        (proper-lists-of-different-length 3 2
                                          ((abc . [1 2 t]) (def . :null)
                                           (abc . [9 :false]))
                                          ((abc . [9 :false]) (def . :null))
                                          first-mismatch-at 0)))
     FAILED   9/21  json-parse-string/object (0.000276 sec) at 
src/json-tests.el:135