Hello everyone!

Here is what happened on 3 Feb 2023 and how we acted to save appdb and all your data.

For those who do not want to read all of this, here is summary:

• appdb was hacked for the first time in its entire history (since 2012)
• intruders were able to encrypt some parts of appdb infrastructure and wanted around 2 bitcoins for decryption of each server
• we have lost IPA cache and MyAppStore library IPA files, as they were not backed up due to high storage costs that we need to cut since PRO cancellation
• we have lost translation website as it wasn't backed up as well (our admin forgot to include it in backup file)
• no outgoing traffic with any data was detected, so nothing was stolen
• user data is safe for two reasons: appdb does not use passwords, only tokens. All appdb actions needs to be confirmed by user on device. If you are very-very-very suspicious, just unlink and link your device to appdb again, new token will be generated.

And now long read, here it comes:

Around 1:30 PM GMT during development on of our team members was unable to access server to deploy new fixes. Then, appdb's database, backend, forums and API went down. Upon investigation of what happened we realized that 3 servers that were hosting appdb's production system, IPA cache and MyAppStore IPA and even backups were compromised on hypervisor level. Hypervisor is an operating system that allows to run virtual machines, simplyfing migration, deployment and development of software. So, on hypervisors we saw stopped virtual machines, stopped services and greeting from hackers stating that we need to send around 2 bitcoins per server to hackers, so they will decrypt and recover our files.

Servers were enemergency booted into recovery environment and we started to investigate what exactly happened.

Upon investigation we found that:

• IPA cache and MyAppStore IPA storage were partially encryted
• backups (despite of they were stored in the same environment) are safe, so we don't need to pay to amazon to recover our files from buckets
• internal cross-datacenter network was completely ruined because of router encryption
• virtual machines that are not encrypted has changed configuration that prevents them from starting
• infrastructure that is responsible for cryptographic credentials managment and issuance is not affected as it is in another data center

Our attempts to fix everything on hypervisor servers were failing - systems were compromised heavily, we were unable to apply patches and upgrade them to safe versions. There was only one way - fix and rescue non-encrypted virual machines and restore from backups the rest.

We did not store backups of MyAppStore IPAs and IPA cache, they were hosted on raid0 arrays. Backuping of 700 TB of data to off-site storage was too expensive for us (appdb is non-profit project, since PRO cancellation we have disabled backup of this part to off-site storage).

So, our team has spent a day by making everything working again.

At the moment, appdb is fully functional. No user data was compromised, as no specific outgoing traffic was recorded and intruders were targeting hypervisors and did not dive inside actual virtual machines.

What's next?

We will optimize our infrastructure and maybe adjust pricing for usage of MyAppStore and IPA cache, so it will cover off-site backups expences. Such cybersecurity incidents are very rare. And current one named in industry ESXiArgs.

Thank you for your patience and support!

Best regards, appdb team.