Starting a cloud migration journey presents exciting opportunities, yet often brings unexpected challenges. Our recent work is a good example. Using the “lift-and-shift” strategy, we’ve successfully migrated most of our workloads. However, some had to be modernized, specifically applications running on WebSphere. The decision to refactor and run these applications on Apache Tomcat was driven by licensing costs, coupled with our discovery that the web applications had minimal dependency on WebSphere-specific components.
We did this change well, put Tomcat on EC2 instances, and place them behind AWS Application Load Balancer (ALB). Of course, enabling end-to-end HTTPS/TLS encryption was a part of our plan to strengthen security.
But after successful testing, Houston got a problem.
Act One: The “Partial HTTPS” Problem
Some applications that needed our system were using HTTP, and couldn’t handle moving from an HTTP endpoint to an HTTPS one, even with redirects.
To accommodate these applications, we created an additional listener in our ALB to serve HTTP.
Our initial plan was to continue maintaining HTTPS between ALB and Tomcat servers. Unfortunately, we hit a snag โ Tomcat treated the connection as secure, sending cookies with the Secure flag. On the flip side, the browser rejected the Secure cookies upon detecting an insecure connection.
Our workaround was to enable two connectors in Tomcat โ one for HTTPS, the other for HTTP, with the intent to eventually deprecate and disable the latter. After all, it made little sense to encrypt requests and responses between ALB and Tomcat if the external connection to ALB wasn’t encrypted.
Act Two: The Cookie Issue
Just as we were recovering from the first issue, we found another anomaly. Tomcat seemed to ignore the JSESSIONID cookie and resorted to URL rewriting. This resulted in the JSESSIONID parameter flying back and forth via URL, while the browser refused to store it in the cookies. The issue only became visible when we faced errors in popup windows that were missing the session identifier in their URLs.
Our troubleshooting efforts were further complicated by the question who is the culprit: is it ALB or Tomcat?
A small hint in Chrome Developer Tools helped us find the root cause. An exclamation mark in a yellow triangle against the Set-Cookie header in the server’s response signaled a bewildering error: This attempt to set a cookie via a Set-Cookie header was blocked because it was not sent over a secure connection and would have overwritten a cookie with the Secure attribute.
Initially, this message left us scratching our heads: why secure connection if we’re using HTTP? Until the pieces fell into place. We used the same browser for testing both endpoints! First, we had tested the endpoint via HTTPS, which stored the cookie with a Secure flag in the browser. Then, when we tested the HTTP endpoint from the same browser, it refused to overwrite the cookie due to the insecure connection. Of course, this issue did not occur in the browser’s incognito mode.
To resolve this, we introduced a filter in Tomcat to rename the cookie from JSESSIONID to JSESSIONID_HTTP for insecure connections, eliminating any confusion between cookies received via different protocols. Meanwhile, we retained the original cookie name for HTTPS connections, optimistic that someday we will completely eliminate HTTP.
So, the testing goes on and we’re looking forward to new surprizes and insights. It’s all part of the adventure of app migrations and modernizations. Bring it on!