Search

Admin Issues

Home
Troubleshooting Guide
Admin Issues

Last Updated: Oct 5, 2021

Articles

I am getting the error "Site cannot be reached" error for the Jiffyweb page.

Image description

  1. Try to telnet from Client machine with jiffy tenant URL and port 443. If it is not connecting, verify the following:
    1. Check if Nginx is running in Core Server. Login to Core server and run command as root user

      ps -ef | grep -i nginx"

    2. If nginx is not running, start the nginx from root user: /opt/nginx/sbin/nginx
    3. If nginx is running, and still user is unable to telnet, check the firewall rules on both Client and Server, then allow the outgoing/incoming traffic appropriately.
  2. If the tenant URL is not resolving from the Client machine, tenant/root URL would not have been added to the Global DNS.
    Add it explicitly to Windows hosts file C:\Windows\System32\drivers\etc and on Core Server under /etc/hosts.
  3. If Tenant URL is resolving and telnet is connecting, open the URL in Incognito window, clear cache, and cookies.

I am getting the error "500 Internal Server Error" for the Jiffyweb page.

  1. Check if Core application is up and running.
    1. Connect to Core server as jiffyapp-usr.
    2. Run the below command to check status of all the processes.

      supervisorctl status

    3. If the processes are up, check the uptime column and make sure no processes are re-starting.
  2. If the application is down and no processes are running, start the application from jiffyapp-usr using the below command.

    application start all

  3. If all processes are running and still issue persists, check the GUS logs for any error and restart GUS, if required.
  4. Check if Postgres DB is down.
    1. Login to DB server and check Postgres process.

      ps -ef | grep -i postgres.

    2. If the Postgres is up, check the connectivity to Postgres from Core server (telnet db_host 5432).
      If that is not connecting, check if there is any firewall blocking on both Core/DB servers.
    3. If Postgres is not up, start postgres from root user.
      • For Jiffy version below 4.2 use:

        systemctl start postgresql-9.6

      • For Jiffy versions above 4.2 use:

        systemctl start postgresql-12

I am getting the error "502 Bad Gateway" after Jiffy login. Image description

If this error shows once you log in to Jiffy Web, it may be due to Anthill is down/hung and GUS is unable to communicate with Anthill.

  1. Check anthill process on Core server as jiffyapp-usr:

    ps -ef | grep -i anthill

  2. If it is not up check for Anthill logs on server/grafana/Kibana and start Anthill:

    supervisorctl start anthill

  3. If the Anthill is up and hung, stop and start the anthill.
    1. Stop anthill:

      supervisorctl stop anthill

    2. After stopping, if the PID still exists, kill it manually using,

      kill -9 {pid}

    3. Start anthill:

      supervisorctl start anthill

I am getting the error "Invalid Subdomain name" while logging into Jiffy. Image description

Invalid Subdomain name error occurs if:

  • Subdomain configuration is not properly provided during the installation.
  • Tenant is not created yet or not configured correctly.

Check the /{JIFFY_MOUNTPOINT}/gus/app.properties, for configured domain and subdomain.
If the subdomain is not updated correctly, update it accordingly and restart the GUS process.

supervisorctl stop GUS

supervisorctl start GUS

  1. Connect to DB server - Connect to GUS DB - Query the tenants table, you will get all the subdomains created for the application. As updated in the app.properties, update it in DB, tenants table as well.
    Query :

    select * from from tenants where tenant_name ='{configured_tenant_name}'; update tenants set sub_domain ='****' where id ='{tenant_id}';

  2. If you don't find the subdomain in the GUS DB - tenants table, you will not be able to login to that sub-domain.
    Please connect with Jiffy_Support for further troubleshooting.

I am getting the error "Unable to retrieve license details" while logging into Jiffy. Image description

  • This issue can happen when the Mongo DB is down, and application is unable to fetch license details from Mongo. Login to DB server and check mongo process: ps -ef | grep -i mongo
    If Mongo process is not running please start the mongo DB process: mongod -f /opt/mongo/conf/mongod.conf
    Post Mongo restart, restart Core application from jiffyapp-usr.
    • application stop all
    • application start all
  • This issue can happen if the Core server time and DB server time are not in sync. As per the jiffy prerequisite, all the servers need to be synced with NTP(Network Time Zone).
    Alternative Solution: Manually sync the times between Core and DB(time difference should not be more then one second)
  • This issue can happen when GUS is unable to connect to Anthill(Anthill connects to Mongo for license details). Could be possible there are some SSL authentication/handshake issues. Check if the java version in core server is >= 1.8.0.275.
    Make sure all the required certificates(mongo_ca, mongo_cert, localhost_ca, localhost_cert) are added to java keystore.

I am getting the error "Could not create vault key" while using Secure Vault.

  1. For any issue with vault, check the vault logs in application core server under $JIFFY_HOME/logs/vault/
  2. This issue happens if, the vault is down. Connect to Core App Server and run the command as jiffyapp-usr:

    vault status

    If it fails to show the status, it means vault is down, you can start the vault with command:

    applicaition start vault

    Once the vault is up and sealed status is true(vault status), then you need to unseal the vault manually with vault keys that were shared during installation, using the below command.

    1. $JIFFY_HOME/.vault.d/vault operator unseal {**********unseal_key_1****************}
    2. $JIFFY_HOME/.vault.d/vault operator unseal {**********unseal_key_2****************}
    3. $JIFFY_HOME/.vault.d/vault operator unseal {**********unseal_key_3****************}

  3. The issue also happens when the vault token is expired, renew the token.
    In case auth token has expired: Renew the auth token.

    vault token renew -increment={timetoLive} {AUTH_TOKEN}

  4. It happens if the localhost certificates that vault uses are expired or the path is not configured correctly. File where certs paths are configured: /JIFFY_HOME/.vault.d/startupconfig.hcl

    After every server reboot, the vault needs to be started manually.

I am getting the error "No matching Machine/Bot running- so aborting execution" in Web UI node. Image description

  1. If you are trying to run with the Developer bot, the developer bot is not up and running. You will have to start the Developer bot and execute the task.
  2. If you are trying to run task with the Execution bot, make sure execution bot is started and the execution bot/cluster is added to the Configuration, and run the task.


This error occurs when the GUS and Anthill connectivity is broken.
Could be possible Anthill is hung and is not able to connect to Mongo DB. Application restart from jiffyapp-usr is required to resolve this issue.

I am getting the error "Failed to parse data sent by Anthill component". Image description

This issue happens when the anthill and mongo connectivity is broken. Restart Core Application.

I am getting the error "HikariPool-1 - Connection is not available,request timedout after 30000ms" in Datasets. Image description

Sometimes the SQL queries would be querying too many records, causing the Drill to get hung. There are three processes that are inter-related Drill, zookeeper, and zeus.

  • Stop all these three process and restart them to resolve this issue as jiffyapp-usr from Core Server.
    Stop processes:
    /opt/docube/apache-drill-1.17.0/bin/drillbit.sh stop
    /opt/docube/zookeeper-3.4.8/bin/zkServer.sh stop
    supervisorctl stop zeus
    Start processess:
    /opt/docube/apache-drill-1.17.0/bin/drillbit.sh start
    /opt/docube/zookeeper-3.4.8/bin/zkServer.sh start
    supervisorctl start zeus

I am getting the error "SYSTEM ERROR:IllegalArgumentException:Attempted to send an message when connection is no longer valid. Query Submission to Drillbit failed". Image description

This issue happens when the drill goes down and there is too much memory consumed by drill.

  1. Check- drill process is up and running. /opt/docube/apache-drill-1.17.0/bin/drillbit.sh status drillbit is not running.
  2. Start the drillbit. /opt/docube/apache-drill-1.17.0/bin/drillbit.sh status drillbit is running. "

I am getting the error “Unable to reach jiffy server, please try after some time . If error persists, please connect Jiffy admin.” while saving task.

This issue occurs if any of the filesystem mount gets fully used.

  1. Login to core server, run command df -Th to see the disk filesystem usage.
  2. Check if any particular mount taking up all space for eg: / (root partition), change dir to /
  3. Run command du -sh * | grep G inside / dir to see which subdirectories are taking up space.
  4. Find the directories and files that consume space and delete unwanted files.
  5. Verify if the configured cleanup scripts are working properly that is scheduled in Cron.

I am getting the error "could not extend file "base/18276/19308.11": No space left on device" during task execution. Image description

This issue happens when there is no space left on DB server.

  1. Login to DB server and check for the filesystem space: df -hT
  2. See if you can reduce any space on DB by deleting unwanted files/logs. If not, increase the DB install mount space.
  3. Also, check if the DB data is fully used by logging in to Postgres/jiffyappdb. See if the db purge is enabled."

Technical Error While Familiarizing the Document using Doc Reader node Image description


This may happen if the server.json file under //docube/server.json is corrupted.

  1. Take backup of existing file try adding new server.json file from Jiffy support and update it accordingly and restart the fileserver process:
    supervisorctl stop fileserver and supervisorctl start fileserver.
  2. This can also happen if the mountpoint between core and cog is not configured properly(For Jiffy 4.4 Patch 8 and below).
  3. Sometimes the folders inside /opt/jiffy/ml would not have full permissions. Give full permissions to /opt/jiffy/ml/output_folder, documentdata etc.

I am getting the error as "System encountered an error" while logging into Jiffy.
Image description

This happens when the redis process is down or when the redis certificates are expired. Check for redis certificate expiry.

  • Nagivate to $DOCUBE_HOME/redis-6.0.5/tests/tls/ dir and run: openssl x509 -text -noout -in redis.crt check the expiry date, if it is expired, renew the certificate and restart the application.


  1. Check the connectivity between Bot Machine and server. Telnet to core server hostname with port 443 from Bot Machine command prompt.
  2. Check load/memory utilization on both Core Application Server and Bot/Client Machine.
  3. Check the Jpipe logs C://Jiffyservice/logs for any connectivity errors.
  4. Check if there is any firewall or antivirus that is blocking the connectivity.
  5. Restart the Jpipe and Jiffy bot to see if the issue resolves.

I am getting the error "java.lang.NullPointerException" while using Doc Reader node. Image description

This happens when the server is upgraded and the client is still in old version.

  1. Login to the RDP, and in the system-tray, right click on the bot icon and click check for updates, if the server is in any higher version, a pop up will appear requesting for an upgrade.
  2. Upgrade the Client and restart the RDP.

Execution Bot in "Not Available" Status.

This issue happens when the Utang process goes out of memory.
You can check the Coral/Zeus logs for out_of_memory errors.
If you find any error in logs for out_of_memory, restart the Utang process from the Core server and restart the bot.

I am getting the error "504 gateway timeout" in Jiffy Web page.

This issue happens when there is too much memory utilization on one of the processes Anthill, Mangrove, Coral, Oreng.

  1. Check for the process that is consuming too much memory, and if it is not releasing the memory, restart that particular process.
  2. If all the process is consuming more space, recommended restarting the application once.
  3. If the swap memory is consumed, recommended to reboot the server.

I am getting the error "Invalid Email Authentication, please check the email settings configured" while configuring Email settings. Image description

  1. This issue occurs when the SMTP/IMAPS servers have issues or if the credentials are wrong.
  2. Connect to Core server and check if the SMTP server host is connecting on the SMTP port using telnet: telnet {email_host} 587 or with port 25 if no TLS is required to connect.
  3. If it is not connecting, allows the SMTP port from core server. And also whitelist the Core server IP so that it allow the traffic from the core Server to SMTP Server.
  4. If it is saying authentication failed, re-check the credentials you are entering.
  5. Issues basically seen with the Email sender node happen due to the Core server Java version not being compatible with the TLS version. Check the GUS logs, and if you see any issues with TLS/SSL_Handshake, please check the Java Version and for further troubleshooting, connect to Jiffy_Support.
  6. Could also happen if any issue with DB sequences is not available or missed. Please connect to Jiffy_Support for this.

I am getting error "Error while connecting to the server" while using Doc reader node. Image description

  1. This issue happens when the shared dir between CORE and ML server is missing(Until 4.4 Patch 8). ML server /opt/jiffy/ml will be mounted on Core server /opt/jiffy/ml
  2. Check if the shared dir is working or the mount is being removed due to some reason. If the nfs is not configured and mount is not present, re-mount the dir. Also, try to telnet from from Core server to Cognitive hostname with port 8765. If it is not connecting check the firewall and try to restart the containers after flushing the ip tables with below commands.
    iptables -t filter -F
    iptables -t filter -X
  3. From Jiffy v4.5 onwards, there is Kubernetes Cluster is introduced. Connectivity with Core to Cluster via port 443 should connect through telnet.
  4. Kubernetes Cluster should be configured properly in the Jiffy Core Application Configuration Files. ($JIFFY_HOME/properties/coral.properties and $DOCUBE_HOME/server.json and /etc/nginx/site_enabled/jiffy_nginx.conf. Check if all pods are running properly in Kubernetes cluster and responding back to core server. Try to telnet from pod to core server with port 443, and also check the pod logs. if any error occures, restart the pod.
Did you find what you were looking for?