Is your company protected against cyberattacks? Have you implemented protective measures, but can’t determine exactly whether they are sufficient and also achieve the desired effect, or where there may still be a need?
In this article, you can learn more about the steps that can help you assess your information security level and what requirements must be met for these steps to result in a useful assessment.
Introduction
As the number of cyberattacks continues to rise, so does the challenge for organizations to protect themselves against such attacks – no easy task, then. Additionally, this endeavour is complicated by the fact that attack methods are constantly changing. Organizations can never rest on the security measures they have already taken in the best-case scenario – because even if one attack can be successfully averted, the next one may already be lurking.
It’s by no means just large companies that are targeted by attackers, where the pickings are supposedly big. Small and medium-sized enterprises (SMEs) are not spared either. Such companies often do not have the resources for extensive investments in their information security and are thus less protected than others. But how can a company determine whether it is protected, regardless of its size? Beyond the selection of concrete protective measures, further questions arise here: Are the measures taken effective? How can it be assessed whether the measures are having the desired effect? Does information security mean that absolutely no security incidents should occur, or are occasional incidents possibly even “normal”? Where does my organization stand in comparison to other organizations in my industry?
As can be seen from these questions, information security is not a simple topic for which there is a universally valid answer. There are many other factors that come into play. For example, specific requirements apply to certain industries. Information security requirements also vary depending on the type and confidentiality of the data being stored or processed. This article attempts to shed light on the aforementioned questions from several perspectives and point out relevant steps for developing a useful benchmark.
A Look at the General State of Information Security
Have you been the victim of a cyberattack? Are you unsure whether such incidents also occur in other organizations or whether something like this only happens at your company? In this case, a look at the figures on the general state of information security can help first: In a 2020 study on white-collar crime [1], about one third of the roughly 5,000 companies surveyed worldwide said they had been the victim of a cyberattack in the past 24 months. Similarly, the study found that cybercrime is one of the three most common offenses committed by white-collar criminals, along with customer fraud and property crimes.
In 2020, the market and social research institute gfs-zürich surveyed 503 managing directors of small Swiss companies on the topic of cybersecurity. [2] This revealed that a successful attack had already occurred in 125 companies. In 34% of these companies, the attack resulted in financial damage, in 10% a damaged reputation was the consequence, and in about 9% customer data was lost. Also, attacks are not limited to specific industries. In 2020, for example, hospitals and healthcare organizations increasingly became the focus of cybercriminals: In Switzerland, the number of attacks on such organizations increased by 59% in November and December 2020, and by as much as 220% in Germany. These figures suggest how present the issue of cyberattacks is and will continue to be in companies and organizations of various sizes and types.
Companies around the world rank cybercrime as one of the biggest business risks for 2021, and of approximately 2,800 survey respondents from 92 countries, 40% said they ranked cyberattacks just behind business interruptions and the outbreak of a pandemic – and that’s in the age of Covid-19. [3]
If we take a closer look at the budgets companies are spending to prevent this risk, we see that the sums are relatively small. According to the “Information Security Benchmark 2019” study [4], which surveyed about 100 participants from various industries, companies invest on average only about 7% of their IT budget in cybersecurity.
This small sample of figures certainly does not paint a comprehensive picture of the information security situation. Nevertheless, the numbers speak for themselves and there are countless other studies and surveys or daily reports that show that cyberattacks pose an increasingly greater risk. Such studies and surveys can help you get a better sense of the overall situation and serve as a guide to where your own organization stands. To return to the question posed at the outset: You are certainly not alone as a victim of a cyberattack.
At this point, it is important not to draw any fatal conclusions: Just because other companies also experience attacks, you should not lull yourself into a false sense of security and assume that an attack is not such a big problem true to the motto “After all, I am not alone”. Instead, this development shows that the need for more security overall is very high. In order not to fall by the wayside as an organization, it is all the more important that you take effective security measures and ensure and monitor their implementation – this is where KPIs come into play as a yardstick.
What Are the Requirements for a Useful Measurement?
In order to establish useful KPIs (Key Performance Indicators) as a benchmark for information security, you must first and foremost determine an information security goal: What does your organization want to achieve in terms of information security? There are several factors to consider, such as the sensitivity of the data your organization processes and stores.
For example, companies in the financial or medical industries are subject to much stricter requirements than organizations that work with less sensitive data. Furthermore, it must be determined whether your organization must comply with regulatory or legal requirements. Appropriate measures must then be developed to implement the security goal set. Here, in addition to the aforementioned binding requirements that may apply depending on the type of organization, general and industry-specific standards and frameworks can be used for guidance. These standards include, for example, ISO standards such as ISO/IEC 27001 for the establishment, implementation, maintenance and continuous improvement of an information security management system (ISMS) [5], including the supplementary guides from the 27000 series of standards, the CIS Controls [6], the NIST Cybersecurity Framework [7], the Swiss ICT Minimum Standard [8] or the BSI standards issued by the German Federal Office for Information Security (BSI) [9], to name just a selection of available guidelines. Although these standards are quite general in parts, they serve as a very good basis for adapting or developing concrete measures that are suitable for your organization.
When selecting and developing measures, the cost-benefit ratio must be taken into account. Of course, the goal should be to achieve the highest possible level of information security, but at the same time, investments should be made in such a way that they are in reasonable proportion to what they can accomplish.
In addition, it is also important to remember that security measures should not unduly restrict the functionality and usability of systems and applications. Otherwise, your systems may be secure, but their intended functionality is no longer fully provided and users are frustrated rather than supportive of the measures. Thus, the actual intention can quickly have an opposite effect. For this reason, it is important to find an appropriate compromise between sufficient security and usability. As an example, consider the implementation of two-factor authentication (2FA): Adding a second factor (e.g., push notification or fingerprint) to the login process to supplement the user password can significantly increase the security of a system. Users do have to perform this second step in addition to entering their password, i.e., confirm a push notification on their smartphone or hold their finger against their device’s fingerprint scanner, but the effort required to do so is rather small – while the effect is comparatively large.
Keep in mind that first a goal and then corresponding measures need to be defined. After all, in order to be able to develop a useful yardstick – in this case the KPIs – it is necessary to clearly define what is to be measured in the first place.
What Should KPIs Look Like and What Are Their Benefits?
Basically, KPIs should fulfill some essential characteristics. These include the following: KPIs should first be clearly understandable and unambiguous. In concrete terms, this means that even non-specialists such as management and employees from other areas can understand what the KPIs say. In addition, KPIs should be measurable, reproducible and actionable. A KPI is reproducible if its determination can be repeated under the same conditions. Implementability in this case means that employees know how they can influence the KPIs or what they can do to achieve the desired values. In addition, KPIs must be consistent with the set goal for information security, as they are designed to help an organization achieve that very goal.
In information security, KPIs are an important tool for measuring and monitoring the effectiveness of implemented security measures. Often, information security is not given the attention it needs until an incident has already occurred – by which time it is usually too late. To counteract this, KPIs should be used, because they make the state of security measurable before an incident occurs. If this is not measurable – and thus, in the case of a solid security level, it is not possible to track how well the measures taken protect the organization – there is a risk that information security will only be seen as a tiresome cost factor. The fact that the measures put in place have already successfully averted incidents only becomes apparent when there is a neutral yardstick: the KPIs. Even in a scenario where the security level is still expandable, without KPIs it is not possible to see where exactly there is still a need to implement or adapt measures.
In addition to assessing information security, such a neutral yardstick is also an essential basis for reporting to management, which is an important component of information security. KPIs also play a role in discussions in which the necessary budgets for security are discussed. With the help of KPIs, the responsible person or department can present management with factual, well-founded and comprehensible information. Conversely, management can use the KPIs to gain a better overview of the relevance of the issue and the current situation without the need for time-consuming research and inquiries.
In addition, cyber fatigue among responsible professionals has been on the rise in recent years. In the “CISO Benchmark Study” [10] conducted by Cisco, 42% of the respondents state that they suffer from cyber fatigue. Cyber fatigue in this context means that they are resigned to proactively defending against possible attacks. This is partly due to the increasing complexity of the environments that study participants manage, and partly due to the reams of security and alert messages they face on a daily basis. 93% of respondents who experience cyber fatigue receive over 5,000 alerts per day. Again, KPIs can counteract this, as they make it easier for employees to determine which events and developments are relevant without having to wade through thousands of notifications.
This makes it all the more clear that compactly summarized, reliable information in the form of KPIs is essential for efficiently evaluating the effectiveness of security measures for all stakeholders.
Where Is the Best Place to Start?
The first step is to determine where your organization even stands in terms of information security. There are different approaches to this. For example, an internal or external audit, which can be based on different standards and methods depending on the requirements and nature of your organization. After the status quo has been determined as a result of the audit, you can start at this point and work out targeted measures to increase the security level – or implement measures recommended during the course of the audit. A gap analysis can also be used to determine the current status and, at the same time, to identify areas where action is still required. Possible weaknesses are identified by comparing this with the target state, which can also be based on various standards. A comparable method is a self-assessment, with the help of which the status of information security can be derived in the form of a checklist or a tool tailored to the standard applied in each case. For example, the German Federal Office for Information Security (BSI) provides such checklists for the IT-Grundschutz Compendium [11] and an assessment tool is also available for the ICT Minimum Standard [12].
Regardless of whether it is an audit, gap analysis or self-assessment – all these checks initially deliver a static result. However, if they are repeated at regular intervals, for example once a year, the weaknesses found and possible improvements can be compared. Building on this, you can apply your KPIs to continuously monitor the state of information security and make dynamic adjustments as needed.
A risk assessment can also provide a basis for evaluating information security and thus a starting point for developing KPIs. Such an assessment results from the probability of occurrence of a risk and the estimation of the potential damage. Thus, it must first be assessed how likely it is that the risk will occur. Similarly, an estimate must be made of the potential damage that an identified risk may cause if it occurs. These estimates usually take the form of classifications, for example, low, medium, and high. Compared to audits, gap analyses and self-assessments, this approach also includes a cost-benefit comparison.
In addition, you should make yourself and your employees generally aware that the responsibility for information security does not lie solely with the CISO, the IT department or certain specialists, but that each individual contributes to it. Many attacks that seem seemingly obvious and not particularly sophisticated still lead to great success, such as phishing campaigns. For this reason, training and awareness of all employees are indispensable pillars of information security.
Is There a Universal Recipe for the Right KPIs?
As can be seen from the start, the right selection of KPIs is highly individual, as there are numerous different influencing factors. Therefore, it stands to reason that there is no universally applicable template that fits all types of organizations and meets all requirements. As an organization, you must determine which KPIs can usefully contribute to meeting your individual information security goal. Again, the standards listed at the beginning should be mentioned here, as they provide a good basis for KPI development and as guides with possible measures and KPIs are available for most of them. For example, there are detailed measures and associated metrics for the CIS Controls. [13] ISO/IEC 27004 [14] serves as a supplementary guide to possible measurement methods for an ISMS in accordance with ISO/IEC 27001. KPIs are by no means about being particularly creative, but about enabling the most efficient and useful ongoing measurement of the state of information security. Therefore, it is advisable to use standards and frameworks that have been proven in practice.
The following are some examples that should not be considered exhaustive and merely serve to give an impression of the versatility of metrics and possible options.
A list of possible metrics is provided, for example, in the “Performance Measurement Guide for Information Security” [15] for NIST Special Publication 800 55 Revision 1. It should be noted that this list is only a basis and must be adapted and expanded as required by your organization. The NIST Guide differentiates between program-level and system-level KPIs. In addition to bare KPIs, it includes additional information such as the overall goal, the formula for determining the KPI, the target value, responsibilities, and data sources, as well as information on at what intervals and in what form the respective KPI should be collected and reported.
The following table provides a selection of the metrics listed therein:
| Category | Metric |
| Security Budget | Proportion (%) of the budget spent on information security compared to the overall information technology budget |
| Vulnerability Management | Proportion (%) of vulnerabilities rated “High” that were remediated within the timeframe defined by your organization after detection compared to identified vulnerabilities rated “High” |
| Training and Awareness | Proportion (%) of information security personnel who have received appropriate security training compared to the total number of information security personnel |
| Maintenance | Proportion (%) of system components that are maintained according to maintenance schedules compared to the total number of system components |
| Identification and Authentication | Proportion (%) of users with access to shared accounts compared to total number of users |
| Contingency Plan | Proportion (%) of systems that have undergone annual contingency plan testing compared to the total number of information systems in the system inventory |
In addition, for example, operational risks can be queried monthly from all department heads in your organization; these include vulnerabilities, errors, downtime, or attacks that may have both occurred or been averted. These are reported with the loss in each case, such as the exact downtime, the number of users that were affected, and the total amount of work time that was lost as a result. The sum of the damage is estimated in the process.
In the context of risks, the number of identified risks with their respective rating (e.g., low, medium, high), which were identified as part of the previously mentioned risk assessment, can also be considered as a KPI.
Like the risk assessment, audits have already been mentioned as a method for determining the status quo of information security. The results of these audits or the resulting to-dos can also be used as KPIs, or more precisely their number. This also includes the number of defined exceptions where controls were not implemented as required by the relevant standard.
For organizations that are more advanced in terms of information security and have implemented a Security Operations Center (SOC) or Security Information and Event Management (SIEM), metrics such as Mean Time to Detection (MTTD) or Mean Time to Remediation (MTTR) are also relevant.
Furthermore, vulnerability scan results can be used, which include, for example, the number of legacy systems and the percentage of patched systems/applications compared to unpatched systems/applications. Tests such as phishing campaigns, tabletop exercises (TTX) and backup recovery tests can also serve as the basis for metrics. The above options give a first impression of the directions that can and should be thought of when introducing and selecting KPIs. It is important that the set of KPIs used is appropriate. If you are only at the beginning of the introduction, you should initially select only a few, which can be gradually expanded or adjusted. Otherwise, the KPIs may miss their target; after all, they are supposed to provide useful and compact information. If there are too many KPIs, it can become difficult to keep track of them and derive actionable insights. In addition, depending on the data required, the collection of KPIs can be costly and cause excessive additional effort.
Conclusion
All in all, KPIs can help make a topic that is difficult to grasp, namely information security, more concrete and thus easier to control. Information security is complex in view of a wide variety of influencing factors, and measuring it is accordingly a complex undertaking.
KPIs can only provide usable information if they are selected carefully. To this end, the basis for useful KPIs must also be in place: appropriate measures. Only then will KPIs show a realistic picture of information security in your organization.
When selecting the measures on which the KPIs are based, the principle that better safe than sorry applies, as it does in general. Thus, measures should not only target incidents that have already occurred, but above all support the prevention of incidents; training is a particularly important component here. Only with solid precautionary measures and continuous monitoring can you ensure that a potential cyberattack does not catch you unprepared.
It may not be in your hands to prevent such attacks from happening in the first place, but you can protect against them and reduce the potential impact if an incident does occur in your organization.
In this case, you should also not show false timidity, but contact appropriate experts or authorities as soon as possible.
About the Author
Lena Mohr joined Oneconsult in July 2020 as Technical Communicator & Team Assistant. She holds a bachelor’s degree in translation (with a focus on technology), is a certified ISO/IEC 27001 Practitioner – Information Security Officer, and has completed a Nano Degree on “Introduction to IT Security”.
[1]: https://www.pwc.de/de/consulting/forensic-services/wirtschaftskriminalitaet-ein-niemals-endender-kampf.pdf
[2]: https://digitalswitzerland.com/wp-content/uploads/2021/02/Schlussbericht_Studie_Digitalsierung_und_CyberriskKMU_2020_12_08.pdf
[3]: https://www.agcs.allianz.com/news-and-insights/news/allianz-risk-barometer-2021-de.html
[4]: https://www.capgemini.com/de-de/resources/information-security-benchmark-2019/
[5]: https://www.beuth.de/de/norm/din-en-iso-iec-27001/269670716
[6]: https://www.cisecurity.org/controls/
[7]: https://www.nist.gov/cyberframework
[8]: https://www.bwl.admin.ch/bwl/de/home/themen/ikt/ikt_minimalstandard.html
[9]: https://www.bsi.bund.de/DE/Themen/Unternehmen-und-Organisationen/Standards-und-Zertifizierung/IT-Grundschutz/BSI-Standards/bsi-standards_node.html
[10]: https://www.cisco.com/c/de_de/products/security/ciso-benchmark-report-2020.html?CCID=cc000160&DTID=odicdc000016&OID=ebksc020457
[11]: https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/Kompendium/checklisten_2019.html
[12]: https://www.bwl.admin.ch/bwl/de/home/themen/ikt/ikt_minimalstandard.html
[13]: https://www.cisecurity.org/white-papers/cis-controls-v7-measures-metrics/
[14]: https://www.beuth.de/de/norm/iso-iec-27004/267539970
[15]: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-55r1.pdf
