I don´t know how you can access to your server, but if you have access to a console (for command line) you can do it w/o rebuild the entire nginx installation.
Can you view/edit the nginx.conf file?
If there have a load_module config parameter that loads the pagespeed module, you only need to delete/comment these line and all pagespeed related config options. (is a dynamic module).
If not (static module) you need to change the executable, but the new one need to be compiled with the same options than the old minus the add-module for pagespeed.
If you do a nginx -V from the command line you got some like:
nginx version: nginx/1.21.0
built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
built with OpenSSL 1.1.1h 22 Sep 2020
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-libatomic --with-openssl=/opt/openssl-1.1.1h --with-openssl-opt=enable-tls1_3 --with-openssl-opt=enable-ec_nistp_64_gcc_128 --add-dynamic-module=/home/builder/rpmbuild/BUILD/nginx-1.21.0/ngx_brotli-1.0.9 --add-dynamic-module=/home/builder/rpmbuild/BUILD/nginx-1.21.0/incubator-pagespeed-ngx-1.14.36.1-beta --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro,-z,now -Os -fpie -fpic -D_GLIBCXX_ASSERTIONS -DTCP_FASTOPEN=23 -Os -flto -Wno-deprecated-declarations -fuse-ld=gold -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC --param=ssp-buffer-size=4 -m64 -mtune=generic -DNGX_HTTP_HEADERS' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie -ljemalloc -lrt
If you need to change the executable these options must be the same in the new.
--prefix=/etc/nginx --sbin-path=/usr/sbin/nginx
--modules-path=/usr/lib64/nginx/modules
--conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid
--lock-path=/var/run/nginx.lock
--http-client-body-temp-path=/var/cache/nginx/client_temp
--http-proxy-temp-path=/var/cache/nginx/proxy_temp
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp
--http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx
--group=nginx
'
As you can see, these config options are path to files/folder nginx uses, including where the nginx.conf is located, so you need the same paths to use actual config files.