Update - We are continuing to monitor for any further issues.
Apr 18, 2024 - 05:20 UTC
Update - Following our latest notice on Saturday April 13th, 2024, we are writing to provide our Customers with an update on the latest available information regarding the Sisense Data Breach.
On Tuesday April 16, 2024, BigPanda was informed by Sisense that the Breach involved Sisense data cubes.
With this new information, we are finalizing a process to export the data that was compromised in the Sisense cubes and provide it to our Customers. We expect this process to be complete by the end of the week, at which time we will be able to schedule individual times upon request.
Our investigation is still ongoing. For any additional requests or to arrange further discussion, please contact the BigPanda Trust & Security team by email at trust@bigpanda.io.
Apr 18, 2024 - 05:19 UTC
Update - BigPanda became aware of a Data Breach through its subprocessor Sisense on Thursday April 11th, 2024. We informed Customers that same day of the Breach via the BigPanda Status Page (status.bigpanda.io). We are writing now to notify our Customers and provide an update on the latest available information.
Incident Summary A Threat Actor compromised the Sisense platform and gained access to BigPanda Customer Data through Snowflake. To our knowledge, the situation has been contained. Our investigation indicates that at no time did the Threat Actor gain access to BigPanda’s Core Platform.
BigPanda’s Use of Sisense and Snowflake The BigPanda Core Platform stores metrics data in Snowflake, which is then loaded into Sisense to deliver metrics reporting via Unified Analytics.
Incident and Data Scope Sisense has confirmed the Breach but has yet to provide detailed information about the incident.
After BigPanda’s internal investigation, it was confirmed that the Threat Actor accessed BigPanda Customer Data stored in Snowflake via Sisense. It was determined that a read-only service account credential for Snowflake was compromised due to the Breach, thus enabling the threat actor to make SQL queries on the Snowflake database.
The BigPanda Customer Data accessed included usernames (which contain BigPanda customer email addresses) and alert and incident tag information. This data in Snowflake enables customers to identify trends within their environments, examples are included below: Username (email address), app_id, source_host, priority, start_time, end_time, date, etc
Incident Containment & Remediation Upon becoming aware of the incident, the BigPanda Security Incident Response Team immediately launched an investigation, which included execution of the following steps: - Requesting information and contacting Sisense; - Reviewing all internal systems audit logs for abnormal activity; - Performing programmatic updates to rotate user access keys and BigPanda’s SSO token via prepared scripts; - Rotating service account keys for systems that feed data to Sisense cubes. - Stopping any new Customer Data from being sent to Sisense. We will re-evaluate this decision in the days to come based on information that will be provided to us from Sisense.
A more detailed list of the actions taken by the BigPanda Security Incident Response team to ensure the containment of the Breach are as follows:
- Changed all Sisense-related passwords on my.sisense.com - For non-Single Sign-On (SSO) access: -- Replaced the Secret in the Base Configuration Security section with your GUID/UUID. -- Reset passwords for all users in the Sisense application. -- Logged out all users by running GET /api/v1/authentication/logout_all under Admin user. - For SSO access: -- Updated sso.shared_secret in Sisense and updated the newly generated value in the SSO handler. -- Rotated the x.509 certificate in our SSO SAML identity provider. -- Rotated the OpenID client secret. -- Updated SSO settings in Sisense with the revised values. -- Logged out all users by running GET /api/v1/authentication/logout_all under Admin user. -- Reset credentials in the database used by the Sisense application.
Ongoing Investigation and Mitigation BigPanda is continuing to investigate the issue and will conduct a full external forensic investigation of its Data Platform. We will continue to provide updates as they become available.
For any additional requests or to arrange further discussion, please contact the BigPanda Trust & Security team by email at trust@bigpanda.io.
Apr 13, 2024 - 20:15 UTC
Monitoring - We want to inform you that we have identified and stopped some suspicious activities from an unidentified threat actor. Pursuing our audit of this security incident, we were able to identify there was read-only access to some user data.
This incident follows a security incident experienced by one of our partners, Sisense. We wanted to let you know that BigPanda takes this matter extremely seriously. We have started a detailed investigation immediately to understand the full extent of what is happening. Our internal investigation determined the data compromise did not include operational data and was limited to analytical data used for our Unified Analytics offering.
As an immediate remediation, we updated all the user access keys and credentials for all our systems. This has terminated access for the threat actor and prevents further intrusion from the recent exploit. We monitored and confirmed there has been no threat actor activity after the user access keys were changed.
The threat actor was able to query a list of usernames, which are email addresses. There is no indication that customer API Keys were compromised since all API Keys are stored in a separate secured environment, which was not compromised as part of this event.
All user access credentials were rotated as of April 11th, 1:45 PM Pacific, and we have completed Sisense recommended procedures.
For any additional requests or to arrange further discussion, please contact BigPanda Security team by email trust@bigpanda.io.
Apr 12, 2024 - 07:08 UTC
Inbound Integrations and Event Processing
Operational
Data Consumption
Operational
Alert Enrichment
Operational
Alert Filtering & Maintenance Plans
Operational
Incident Enrichment & Environments Population
Operational
Completed -
The scheduled maintenance has been completed.
Apr 14, 11:50 UTC
Verifying -
Verification is currently underway for the maintenance items.
Apr 14, 09:44 UTC
In progress -
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 14, 07:30 UTC
Scheduled -
In support of our commitment towards improving our services and infrastructure, we will be performing the following maintenance to the BigPanda platform.
Start Time: April 14, 2024 07:30 UTC End Time: April 14, 2024 13:30 UTC Duration: 6 hours
What's Happening?
During this maintenance, we will be upgrading our Mongo cluster from v5.0.24 to v6.0.13.
During the maintenance window - as most BigPanda services use Mongo, customers may expect up to 45 minutes of latencies within the BigPanda Pipeline services while we initiate service restarts to remediate experienced connectivity issues.
We will monitor our pipeline closely as changes are deployed.
Is User Action Required?
No user action is required.
The planned maintenance has successfully completed our QA validation and regression testing. We will be closely monitoring the maintenance work, and BigPanda’s status page will be updated throughout the maintenance period with any changes to service operations.
We apologize for any inconvenience that this may cause. If you have any questions, please reach out to BigPanda Support via our live in-app chat or via support@bigpanda.io.
Apr 5, 21:06 UTC
Completed -
The scheduled maintenance has been completed.
Apr 11, 15:36 UTC
In progress -
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 11, 15:00 UTC
Scheduled -
We want to inform you that we will be performing the following maintenance to the BigPanda platform:
Start Time: April 10, 2024 15:00 UTC End Time: April 10, 2024 18:00 UTC Duration: 3 hours
What's Happening?
During this maintenance, BigPanda will be performing updates to align access controls with our Zero Trust Network Architecture initiative. Specific security groups will be updated to improve operational access restrictions.
We will be closely monitoring the maintenance work, and BigPanda’s status page will be updated throughout the maintenance period with any changes to service operations. We apologize for any inconvenience that this may cause. If you have any questions, please reach out to BigPanda Support via our live in-app chat or via support@bigpanda.io.
Apr 8, 19:36 UTC