No clue... I'm totally unfamiliar with Xemacs. Likely it just has slightly different syntax for syntax-tables. I think that line was copied from lisp.el, though... maybe whatever Xemacs' equivalent file is has a compatible version.
The only other reference to "modify-syntax-entry" I found was in slime.el, both for Emacs and XEmacs. lisp.el doesn't have it. There were only two "modify-syntax-entry" lines used, vs. the many in arc.el. Lines 6451-6462 from slime.el:
(defvar sldb-mode-syntax-table
(let ((table (copy-syntax-table lisp-mode-syntax-table)))
;; We give < and > parenthesis syntax, so that #< ... > is treated
;; as a balanced expression. This enables autodoc-mode to match
;; #<unreadable> actual arguments in the backtraces with formal
;; arguments of the function. (For Lisp mode, this is not
;; desirable, since we do not wish to get a mismatched paren
;; highlighted everytime we type < or >.)
(modify-syntax-entry ?< "(" table)
(modify-syntax-entry ?> ")" table)
table)
"Syntax table for SLDB mode.")
