Skip to main content

Website

Regarding outage issues with cmass.org

Summary: The latest from ClubExpress. Unclear from this message is whether we're over this - the note about "stable functionality on 06/25/2026" has not aligned with some members experience here.


================


Dear ClubExpress Admins,


As noted in our prior communication, ClubExpress recently became aware of disruptions involving our platform. In response, we quickly took steps with the assistance of outside technical resources, to ensure the security of the platform and investigate the cause of the disruption. Through the diligent efforts of our internal and external resources we were able to securely restore our platform to stable functionality on 06/25/2026.


Although our investigation into the full nature and scope of the disruption remains ongoing, we determined that the disruption was caused by a large increase of traffic to the platform. We can provide relevant updates should our investigation uncover any substantive changes. We will follow up as we learn more. 


Regards, 

Geoffrey Walsh

Director of CS Operations

Summary: Progress being made; working with AWS; not yet out of the woods; not at a point where CE can provide a complete picture as to what's going on


====================



Dear ClubExpress Community,


We want to acknowledge what many of you have experienced over the past two weeks; intermittent slowness and availability issues that have disrupted your ability to use the platform as you depend on it. We want to give you a straightforward update on where things stand.


Our engineering team, working alongside AWS and outside technical resources, has been focused entirely on stabilizing the platform and addressing the underlying causes of these issues. Progress is being made, and that work continues around the clock.


We are not yet at a point where we can provide a complete picture, and we will not communicate prematurely. What we can commit to is this: we will share additional updates as we have accurate and reliable information to provide.


We recognize this has been a difficult two weeks for many of your organizations. We appreciate your patience and will continue to keep you informed.


On some positive notes, the issue with uploading documents and images has been resolved. Also anyone using ad hoc reporting you are no longer receiving an error and should be able to run reports.


Regards, 

Geoffrey Walsh

Director of CS Operations

Latest. Executive summary: things are getting incrementally better, slowly.

≠=≠==========================


Dear Club Admin,

 

We wanted to share an update on the progress being made, for the ongoing disruption with ClubExpress. 

 

As previously communicated, our systems continue to receive thousands of illegitimate requests designed to overwhelm our infrastructure and impact normal service. This activity remains the primary cause of the intermittent slowdowns and availability issues some members have experienced. 


Over the past several days, our engineering team has made meaningful progress:


Multiple attack patterns have been identified and analyzed.

Additional filtering and mitigation rules have been deployed to block invalid traffic before it reaches our application servers.

Infrastructure adjustments have improved our ability to absorb attack traffic while maintaining service for legitimate users.

We are seeing periods of improved stability and faster response times compared to earlier stages of the incident.


While these measures are helping, the attack remains active and continues to evolve. Our team is adjusting defenses in real time as new traffic patterns emerge. We are monitoring the platform around the clock and deploying additional protections as needed.


We know this disruption has been frustrating and has affected your organization's operations. We appreciate the patience, understanding, and feedback many of you have provided throughout this event. Your reports have helped us identify areas requiring additional attention and have guided some of our mitigation efforts.


Our current focus remains:


Maintaining platform availability for legitimate users.

Further reducing the impact of malicious traffic.

Continuing performance and stability improvements across all services.

Restoring normal operations as quickly and safely as possible.


We will continue providing regular updates as significant progress is made. While we are encouraged by the improvements we are seeing, we will remain fully engaged until service performance has returned to expected levels. We recognize that nine days of intermittent disruption is an unacceptable experience, and we do not take lightly the impact this has had on your clubs, members, and operations.


This incident has the full attention of ClubExpress executive leadership, who are actively engaged alongside our engineering, infrastructure, and support teams. Resolving this disruption is our organization's singular priority until service has been fully restored.


Regards, 

Geoffrey Walsh

Director of CS Operations

Problems are ongoing - latest message from ClubExpress:

=========================================


Dear Club Admin,

 

I wanted to provide a brief update regarding the ongoing site performance issues. Please know that this remains our highest priority, and our teams continue to actively investigate and address issues as they arise.

 

We have made meaningful progress since last week and have resolved several underlying issues. While those efforts have improved overall stability, we continue to identify and work through new issues each day as we address the remaining impacts.

 

Our teams are monitoring site performance closely and dedicating all necessary resources to improving stability and the overall user experience. We appreciate your patience and understanding as we continue this work.

 

I will continue to provide updates as soon as I have additional information to share.

 

Warm regards,

ClubExpress Team

Dear Club Admin,

 

We want to follow up on the performance issues many of you experienced this week and share an update on the steps our engineering team has taken to stabilize the platform.

 

We have been working around the clock, and we are pleased to report that significant improvements have been made. Here is a summary of what has been put into place:

 

• Upgraded server hardware – We doubled the processing power and RAM on each of our web servers, giving them greater capacity to handle traffic spikes without degradation in performance.

 

• Expanded our server infrastructure – We added a third web server to distribute load more evenly across the environment. This has meaningfully leveled out resource usage and contributed to the stability improvements you may already be noticing.

 

• Implemented a Web Application Firewall (WAF) – We added a new layer of protection between our front-end load balancer and the application servers. This allows us to filter out malformed, and high-volume automated requests before they ever reach the application — freeing our servers to focus on handling legitimate traffic from you and your members.

Resolved a URL handling issue – We identified and fixed a bug that was causing the platform to respond to invalid URLs in a way that inadvertently encouraged search engine bots to continue scanning them repeatedly. The fix ensures those requests are now properly rejected, which will reduce bot-driven traffic to the platform over time.

 

Taken together, these changes are having a real impact. We believe we are through the most difficult period, and system performance should continue to improve in the days ahead.

 

A note going forward: As we fine-tune some of the new filtering rules, there is a possibility you may encounter occasional unexpected behavior on specific platform features. If you notice anything that does not seem to be working correctly, please reach out to our support team so we can investigate promptly. We appreciate your patience as we complete this calibration work.

 

We also have additional improvements underway — including further infrastructure expansion and application-level optimizations — that we will continue to roll out in the coming weeks.

 

Thank you for your continued trust in ClubExpress. We do not take lightly the role our platform plays in keeping your clubs and organizations running, and we are committed to delivering the reliability you deserve.

 

Warm regards,

ClubExpress Team

Dear Club Admin,

 

We want to keep you informed about the work currently underway to improve the performance and stability of the ClubExpress platform. Our engineering team has several efforts in progress, and we wanted to share an update on where things stand.

 

Here is what we are actively working on:

 

• Server capacity expansion – We are acquiring additional licenses to scale out our web farm infrastructure, which will allow us to add more servers and better distribute load across the platform.

 

• Caching optimization – We are evaluating how caching is currently utilized across the system to identify opportunities to reduce CPU demand and improve overall efficiency.

 

• Enhanced traffic filtering – We are implementing additional rules within our Web Application Firewall (WAF) to more effectively block invalid or malicious traffic before it reaches our servers.

 

• Request processing improvements – We are reviewing the request handling pipeline to identify bottlenecks and ensure that invalid requests are failed quickly, freeing up resources for legitimate traffic.

 

• Improved crawler behavior – We recently deployed a fix that now returns a proper 404 (Not Found) response for invalid URLs, rather than redirecting to the site home page. This signals to search engine crawlers that those URLs do not exist, reducing unnecessary repeat crawling and the load it places on our servers.

 

We also have a number of additional items under investigation that we will share more about as they progress.

 

We appreciate your patience as we work through these improvements. Our goal is a faster, more reliable experience for you and your members, and we are committed to keeping you informed along the way.

 

Regards, 

ClubExpress Team

Dear Club Admin,

 

We wanted to provide you with an update on the service disruption currently affecting our platform.


Our systems are experiencing a significant surge of invalid traffic - thousands of illegitimate requests flooding our servers in a short period of time. This activity is straining system performance and is the primary cause of the intermittent availability issues you may be encountering.


Our engineering team has identified several patterns within this traffic and is actively deploying filtering measures to reduce its impact. While these steps are showing early signs of progress, we have not yet fully resolved the issue.


We are seeing gradual stabilization, and our team is working around the clock to restore consistent performance as quickly as possible. We will continue to share updates as the situation develops.


We sincerely appreciate your patience and understanding.

 

Regards, 

ClubExpress Team

ClubExpress

info@clubexpress.com


Dear Club Admin,


We're following up on the message we sent Friday regarding website performance issues. We know many of you are starting your week with events to manage, members to communicate with, and registrations to process, and we want you to know we're working just as urgently today as we were on Friday to get things back to normal.


Here's what we know: the disruption is being caused by a distributed denial-of-service (DDoS) attack, which is essentially an attempt to overwhelm our systems with excessive traffic and make sites slower to load. This type of attack affects site availability; it does not compromise data. Your member information, payment data, and account credentials remain fully secure.


Our team continues to actively work on mitigation efforts and we're making progress, but we're not fully resolved yet. We'll send another update as soon as we have a meaningful change in status to share. We do not have an ETA at this time. 


Thank you for bearing with us. Thank you for your continued support and understanding.

 

Regards, 

ClubExpress Team

arrow_backReturn to Forum

CMASS is a chartered section (#464) of the National Association of Rocketry.