Search

Jiffy Upgrade

Home
Getting Started
installation steps
jiffy upgrade

Last Updated: May 10, 2022

Articles

Supported Upgrade Versions

The following Jiffy versions can do a direct upgrade to version 4.7.

  1. Jiffy Automate 4.6.0
  2. Jiffy Automate 4.6.1
  3. Jiffy Automate 4.6.2
  4. Jiffy Automate 4.6.3
  5. Jiffy Automate 4.6.4
  6. Jiffy Automate 4.6.5
  7. Jiffy Automate 4.6.6
  8. Jiffy Automate 4.6.7

Artifact list

Name Version Type Description
jiffy-playbook.tar.gz 4.7 Ansible playbook Ansible scripts which does the upgrade
jiffy-upgrade.tar.gz 4.7 Compressed tar Jiffy core artifacts
jiffy-helms-v.4.7.zip 4.7 Helm charts Helm charts for container deployments

Container List

Container Tags (4.7) Tags (4.7.1) Tags (4.7.2)
backward-comp JARVIS-MAIN-BackwardCompatibilityEngine6419Apr22 JARVIS-MAIN-4.7-BackwardCompatibilityEngine6624May22
bolcategory JARVIS-MAIN-vendorcategorisation_11026Apr22 JARVIS-MAIN-4.7-vendorcategorisation_11726May22
bolml JARVIS-MAIN-T4.3.017526Apr22 JARVIS-MAIN-4.7-T4.3.017824May22
docsplit JARVIS-MAIN-DocumentSplit_6026Apr22 JARVIS-MAIN-4.7-DocumentSplit_6624May22
functionator JARVIS-MAIN-232 JARVIS-MAIN-4.7-238
handwritingsegmentation JARVIS-MAIN-HWSegmentation_11927Apr22 JARVIS-MAIN-4.7-HWSegmentation_12224May22
invoicecategory JARVIS-MAIN-T4.3.017526Apr22 JARVIS-MAIN-4.7-T4.3.017824May22
invoiceml JARVIS-MAIN-T4.3.017526Apr22 JARVIS-MAIN-4.7-T4.3.017824May22
pdf2image JARVIS-MAIN-Pdf2Image_7726Apr22 JARVIS-MAIN-4.7-Pdf2Image_7924May22
pdf2json-service JARVIS-MAIN-Pdf2JsonService_10725Apr22 JARVIS-MAIN-4.7-Pdf2JsonService_11231May22
pdfsplit JARVIS-MAIN-Pdfsplit_6919Apr22 JARVIS-MAIN-4.7-Pdfsplit_7124May22
portfolio JARVIS-MAIN-4.7-Portfolio_27902Jun22 JARVIS-MAIN-4.7-Portfolio_28514Jun22 JARVIS-MAIN-4.7-Portfolio_27902Jun22
template-converter JARVIS-MAIN-198 JARVIS-MAIN-4.7-209
w2ml JARVIS-MAIN-w2ml_8226Apr22 JARVIS-MAIN-4.7-w2ml_9125May22
w2split JARVIS-MAIN-W2Split_10126Apr22 JARVIS-MAIN-4.7-W2Split_10624May22
lineitemmatch JARVIS-MAIN-LineItemMatch_7226Apr22 JARVIS-MAIN-4.7-LineItemMatch_7724May22

No changes are made to the helm charts after release 4.6.

TCP Ports

Component Port Process Source Destination
Email Server (TCP Ports) 25 SMTP Server Core Application Server
Email Server 993 IMAP/IMAPS Core Application Server
Jiffy Web 443 Nginx Any windows machine where the user accesses Jiffy application Core Application Server
JPipe gRPC 443 Nginx All the windows client machines where BOT is installed Core Application Server
RabbitMQ Viewer 15672 RabbitMQ Any windows machine where the user accesses RabbitMQ web page Core Application Server
MongoDB 27017 MongoDB Core Application Server Database Server
Postgres 5432 Postgres Core Application Server Database Server
Kubernetes Server 443 Nginx Ingress Core Application Server Kubernetes Server
LDAP Server 389 LDAP Core application server LDAP Server

Third Party Packages Installed/Upgraded

Software Existing Version Latest Version
Vault 1.8.5 1.10.0
Drill 1.17 1.19.0
Nginx 1.17.8 1.20.2
libedit-devel NA 3.0-12.20121213
Python 3.7.9 3.10.3
ZooKeeper 3.4.8 3.7.0
RabbitMQ 3.8.2 3.9.14-1
Mongo 4.0.19-1 4.2.19
.Net Framework 4.7.2 4.8 +
llvm5.0-devel NA 5.0.1-7
llvm-toolset-7-clang NA 5.0.1-4
Redis 6.0.5 6.2.6
Postgres - DB server version 12.4 14.2
erlang 22.3.4.6-1 23.3.4.11-1
libicu NA 50.2-4

Helm Chart Changes

Chart change Affected Charts Changes
Removed .ht from jiffy-auth kubernetess secret
  • backward-comp
  • bolml
  • bol_category
  • docsplit
  • handwritingsegmentation
  • invoiceml
  • invoicecategory
  • pdf2image
  • pdf2json-service
  • pdfsplit
  • portfolio
  • w2ml
  • w2split
  • lineitemmatch
 templates/deployment.yaml
Image description
MASTERKEY is now readable from the pod environment variables instead of reading from file inside the pod.
  • backward-comp
  • bolml
  • bol_category
  • docsplit
  • handwritingsegmentation
  • invoiceml
  • invoicecategory
  • pdf2image
  • pdf2json-service
  • pdfsplit
  • template-converter
  • portfolio
  • w2ml
  • w2split
  • lineitemmatch
  • functionator
 templates/deployment.yaml
Image description
pod configured to use jiffy-auth secret
  • functionator
  • template-converter
pod configured to use services-properties configmap
  • template-converter

Upgrade Prerequisites

  1. Stop Jiffy application.
    1. Log in to the core server shell.
    2. Switch to jiffy application user.
    3. Stop the application.

      1. application stop all
      2. application stop vault

  2. Backup the Postgresql, and MongoDB databases.
    1. For Jiffy and GUS postgres backups (Commands will prompt for DB password):

      1. pg_dump -h {DB hostname} -p 5432 {jiffy-schema-name} -U {username} | gzip > {filename.gz}
      2. pg_dump -h {DB hostname} -p 5432 {gus-schema-name} -U {username} | gzip > {filename.gz}

    2. For mongoDB backup
      Self Hosted:
      1. Connect to the mongoDB server
      2. Switch to jiffy application user

        mongodump --ssl --host {MongoDB hostname} --port 27017 --sslPEMKeyFile {Keyfile} --sslAllowInvalidCertificates --authenticationDatabase admin -u admin --out {filename.dump}

        Command will prompt for DB admin password.
  3. Backup Postgresql Data directory, MongoDB Data directory, Vault data directory

Pre-Upgrade steps

  1. Connect to core server.
  2. Switch to the jiffy application user.
  3. Download the artifact files:
    1. Download helm charts.

      1. export JIFFY_INSTALL=“/tmp/jiffy-install/4.7/”
      2. mkdir -p $JIFFY_INSTALL/helm/
      3. cd $JIFFY_INSTALL/helm/
      4. wget –user {username} –ask-password downloads.jiffy.ai/4.7/Upgrade/jiffy-helms-v4.7.zip

    2. Extract the helm charts in jiffy core server.

      unzip jiffy-helms-v4.7.zip

    3. Download ansible-playbook and upgrade artifact.

      1. cd $JIFFY_INSTALL
      2. wget –user {username} –ask-password downloads.jiffy.ai/4.7/Upgrade/jiffy-playbook.tar.gz
      3. wget –user {username} –ask-password downloads.jiffy.ai/4.7/Upgrade/jiffy-upgrade.tar.gz

    4. Extract ansible-playbook and upgrade artifact.

      1. tar -xf jiffy-playbook.tar.gz
      2. tar -xf jiffy-upgrade.tar.gz -C $JIFFY_INSTALL/upgrade/

  4. Fill in the inventory file and variable.yml file as per requirement (use the existing files or parameters from the previous/last installation as reference)
    1. The inventory file will have all the details of the servers, authentication methods, etc.
    2. variable.yml file contains all the parameters such as the username, custom application parameters, mount point, etc.

Ansible ENV Creation and YUM Updates

For RHEL (as root user):

The pip package pyscopg requires yum package postgresql-devel.

The postgresql-devel yum package requires llvm-toolset-7-clang yum.

To install llvm-toolset-7-clang and to enable repository, use the following commands.

  1. subscription-manager repos --enable rhel-7-server-devtools-rpms
  2. subscription-manager repos --enable rhel-server-rhscl-7-rpms
  3. cd /etc/pki/rpm-gpg
  4. wget -O RPM-GPG-KEY-redhat-devel https://www.redhat.com/security/data/a5787476.txt
  5. rpm --import RPM-GPG-KEY-redhat-devel
  6. yum install http://mirror.centos.org/centos/7/os/x86_64/Packages/libedit-devel-3.0-12.20121213cvs.el7.x86_64.rpm
  7. yum install llvm5.0-devel -y
  8. yum install llvm-toolset-7-clang -y
  9. yum install libicu -y

Common steps (as root user):

  1. export JIFFY_INSTALL="/tmp/jiffy-install/4.7/"
  2. cd $JIFFY_INSTALL/upgrade/
  3. chmod u+x presetup.sh
  4. ./presetup.sh

Backup

The following steps will cover code backup, in case of rollback to a previously installed version.

  1. Activate the Ansible python virtual environment.
    Applicable only, if Ansible is installed under python virtual environment.

    source /opt/jiffy3rdparty/ansibleEnv/bin/activate

  2. Start backup process:

    1. cd $JIFFY_INSTALL
    2. ansible-playbook pre-upgrade/root.yml -i inventory -e @variable.yml

    For debug mode

    ansible-playbook pre-upgrade/root.yml -i inventory -e @variable.yml -v

    For detailed debug mode:

    ansible-playbook pre-upgrade/root.yml -i inventory -e @variable.yml -vvvv

  3. Switch to the jiffy application user
  4. Activate the ansible python virtual environment.
    Applicable only, if ansible is installed under python virtual environment

    source /opt/jiffy3rdparty/ansibleEnv/bin/activate

  5. Run the non-root installer : (Sample variable and inventory file below)

    ansible-playbook pre-upgrade/non-root.yml -i inventory -e @variable.yml

    For debug mode:

    ansible-playbook pre-upgrade/non-root.yml -i inventory -e @variable.yml -v

    For detailed debug mode:

    ansible-playbook pre-upgrade/non-root.yml -i inventory -e @variable.yml -vvvv

Core Upgrade

  1. Switch to root user in the jiffy core server.
  2. Activate the ansible python virtual environment.
    Applicable only, if ansible is installed under python virtual environment.

    source /opt/jiffy3rdparty/ansibleEnv/bin/activate

  3. Execute the playbook to begin the upgrade.

    1. export JIFFY_INSTALL="/tmp/jiffy-install/4.7/"
    2. cd $JIFFY_INSTALL
    3. ansible-playbook upgrade/root.yml -i inventory -l core -e @variable.yml

    For debug mode:

    ansible-playbook upgrade/root.yml -i inventory -l core -e @variable.yml -v

    For detailed debug mode:

    ansible-playbook upgrade/root.yml -i inventory -l core -e @variable.yml -vvvv

    For sudo and ssh password prompt add -kK options along with run command:

    ansible-playbook upgrade/root.yml -i inventory -l core -e @variable.yml -kK

  4. Change ownership of /tmp/jiffy-install to jiffy application user.

    chown -R {jiffyapp linux user}:{jiffyapp linux user} /tmp/{jiffy-install}

  5. Switch to the jiffy application user.
  6. Activate the ansible python virtual environment.
    Applicable only, if ansible is installed under python virtual environment.

    source /opt/jiffy3rdparty/ansibleEnv/bin/activate

  7. Run the non-root installer : (Sample variable and inventory file below)

    1. export JIFFY_INSTALL="/tmp/jiffy-install/4.7/"
    2. cd $JIFFY_INSTALL
    3. ansible-playbook upgrade/non-root.yml -i inventory -l core -e @variable.yml

    For debug mode:

    ansible-playbook upgrade/non-root.yml -i inventory -l core -e @variable.yml -v

    For detailed debug mode:

    ansible-playbook upgrade/non-root.yml -i inventory -l core -e @variable.yml -vvvv

  8. Execute the below ansible command to update application configuration changes.
    with masterkey prompt:

    ansible-playbook jiffyconfiguration.yml -l core -e @variable.yml

    without masterkey prompt:

    ansible-playbook jiffyconfiguration.yml -l core -e masterKey='{replaceme}' -e @variable.yml

  9. Start and unseal vault.

    1. cd $JIFFY_HOME/.vault.d/
    2. nohup vault server -config=startupconfig.hcl >vault_$(date +%Y%m%d_%H%M%S).log 2>&1 &
    3. vault operator unseal {unseal key 1}
    4. vault operator unseal {unseal key 2}

DB Upgrade

During this upgrade, ulimits value is modified on the DB server.

  1. Switch to root user in the core server
  2. Activate the ansible python virtual environment.
    Applicable only, if ansible is installed under python virtual environment.

    source /opt/jiffy3rdparty/ansibleEnv/bin/activate

  3. Execute the playbook as root/sudo user to begin the upgrade.

    1. export JIFFY_INSTALL=“/tmp/jiffy-install/4.7/”
    2. cd $JIFFY_INSTALL
    3. ansible-playbook upgrade/root.yml -i inventory -l db -e @variable.yml

    For debug mode:

    ansible-playbook upgrade/root.yml -i inventory -l db -e @variable.yml -v

    For detailed debug mode:

    ansible-playbook upgrade/root.yml -i inventory -l db -e @variable.yml -vvvv

Kubernetes Upgrade

  1. Connect to Core server.
  2. Execute the following commands.

    1. export JIFFY_INSTALL="/tmp/jiffy-install/4.7/"
    2. cd $JIFFY_INSTALL/helm/
    3. chmod 777 k8s-upgrade.sh

  3. Switch to jiffy application user.
  4. Remove old jiffy-auth secrets.

    kubectl delete secret jiffy-auth -n jiffy-cognitive

  5. Create jiffy-auth secrets.

    kubectl create secret generic jiffy-auth --from-literal=password_key='{masterkey}' --from-file={mountpoint}/jiffy/auth/.at -n jiffy-cognitive

  6. Execute the script for Kubernetes upgrade. Default values for the Kubernetes upgrade script.

    • namespace(n)=jiffy-cognitive
    • replicacount(r)=1
    • mountpoint(m)=/opt

  7. Uninstall the existing docplit container from the cluster.

    helm uninstall docplit -n {namespace}

    ./k8s-upgrade.sh -n {namespace} -c {clusterDNS|mandatory} -r {replicacount} -m {mountpoint}

Post upgrade

Restart Nginx:

Nginx on core server has to be restarted by user with elevated privileges.

  • /opt/nginx/sbin/nginx -s stop
  • /opt/jiffy3rdparty/nginx/sbin/nginx

Drill HEAP and MAX Memory

As jiffy application user, execute the below commands to modify/apply DRILL_MEMORY parameters.

  • #!/bin/bash
  • sed -i 's/^#export DRILL_HEAP=${DRILL_HEAP:-"4G"}/export DRILL_HEAP=${DRILL_HEAP:-"8G"}/g' $DOCUBE_HOME/apache-drill-1.17.0/conf/drill-env.sh
  • sed -i 's/#export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}/export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"16G"}/g' $DOCUBE_HOME/apache-drill-1.17.0/conf/drill-env.sh
  • application stop all
  • application start all

Verify the DRILL_MAX_HEAP_MEMORY on the drill configuration file.

cat $DOCUBE_HOME/drill/conf/drill-env.sh

Testing

Verification:

  • Connect to core server.
  • Switch to jiffy application user.
  1. Verify jiffy version by running the following command in the core server. Version should be 4.7

    cat $JIFFY_HOME/productversion.txt

  2. Verify jiffy client version by running the following command in the core server. Version should be 4.7

    cat $JIFFY_HOME/data/client/setup/version.txt

  3. Verify configuration in the application properties file.
    1. anthill.properties ($JIFFY_HOME/properties/)
    2. mangrove.properties ($JIFFY_HOME/properties/)
    3. coral.properties ($JIFFY_HOME/properties/)
    4. application.properties ($GUS_HOME/)
    5. app.properties ($JIFFY_HOME/)
  4. Check uptime and status of jiffy core application processes

    supervisorctl status

  5. Check pod status.

    kubectl get pods -n {namespace}

  6. Check Vault status.

    vault status

Smoke Test

  1. Connect to Bot Machine.
    1. Right-click on the Bot Manager bottom right and Press **Check for updates**.
    2. Click the **check for updates** button to upgrade bot machine.
  2. Connect to jiffy application URL.
  3. Log in to tenant.
    1. Run the below tasks (if available):
      1. Doc node
      2. Web node
      3. Excel node
      4. Add new secrets to vault and verify vault status.
      5. Add dataset

See Also

Did you find what you were looking for?