Complying with the NERC CIP requirements is expected to be a major expense for power producers and the like in the coming years. To date, companies have spent tens of millions of dollars to formally document and test the support for internal control assertions required by CIP and maintaining this documentation will continue to be costly beyond the first round of documentation.

Let’s take a look at some of the most important components of a good NERC CIP compliance program:

Automate The Testing & Reporting Of All Of The Technical Controls

An important concern for power producers is finding a cost-effective method of documenting, storing, and analyzing CIP control assessment work. Management is also looking to meet all the technical requirements spelled out in the requirements. In the first year, many companies have elected to use spreadsheets to tackle CIP documentation because they are familiar with the tool. Moreover, some companies prefer to use spreadsheets because the CIP requirements are still evolving.

Spreadsheets have significant limitations that will increase compliance risks. In addition, depending on spreadsheets for CIP documentation may prevent companies from improving their compliance process and risk management capabilities. In the first year of CIP compliance efforts, many internal auditors and project consultants have advised power producers to use their existing spreadsheet software to document compliance efforts. There is no way that these tools are sufficient to document all relevant accounts, account assertions, risks, controls, and deficiencies. The only way to truly document everything is to automate the process.

Use File Integrity Checks To Assure Your Systems Are In A Desired State

It is very difficult to compromise a system without altering a system file, so file integrity checkers are important. A file integrity checker computes a checksum for every guarded file and stores it. At a later time you can compute a checksum again and test the current value against the stored value to determine if the file has been modified. Some lesser quality file integrity checkers use a 32 bit CRC (Cyclic Redundancy Check). Attackers have demonstrated the ability to modify a file in ways that the CRC checksum could not detect, so stronger checksums known as cryptographic hashes are recommended.

One of the challenges in using a file integrity checker is the false positive problem. When you update files or apply system patches this changes the file. Creating the initial database of signatures is easy; keeping it up to date is much harder. However, even if you only run the checker once (when you first install the system) this can still be very valuable. If you are ever concerned that the system was compromised you can run the checker again to determine which files have or have not been modified.

The other challenge with a file integrity checker is that you have to have a pristine system when you create the first reference database, otherwise you may be creating cryptographic hashes of a compromised system while feeling warm and fuzzy that you are implementing good security. It is also very important that you store the reference database offline or an attacker may be able to compromise the system and hide their tracks by modifying the reference database.

Test System Configurations Against External & Internal Policies

Testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Testing also provides an objective, independent view of the configurations to allow the facility to understand the risks in the implementation of the configurations. Test techniques include, but are not limited to, the process of executing a configuration with the intent of finding “bugs”.

Testing can also be stated as the process of validating and verifying that the configurations:

• meets the business and technical requirements that guided its design and development;
• works as expected; and
• can be implemented with the same characteristics.

Testing, depending on the testing method employed, can be implemented at any time in the development process. However, most of the test effort occurs after the requirements have been defined and the coding process has been completed. As such, the methodology of the test is governed by the configuration methodology adopted.

Testing can never completely identify all the defects within your configurations. Instead, it furnishes a criticism or comparison that compares the state and behavior of the configurations against principles or mechanisms by which someone might recognize a problem. These principals or mechanisms may include (but are not limited to) specifications, contracts, comparable products, past versions of the same product, inferences about intended or expected purpose, user or customer expectations, relevant standards, applicable laws, or other criteria.

A study conducted by NIST in 2002 reports that bugs cost the U.S. economy $59.5 billion annually. More than a third of this cost could be avoided if better testing was performed.

A primary purpose for testing is to detect configuration failures so that defects may be discovered and corrected. This is a non-trivial pursuit. Testing cannot establish that configurations functions properly under all conditions but can only establish that they do not function properly under specific conditions. The scope of testing often includes examination of configurations as well as execution of those configurations in various environments and conditions: does it do what it is supposed to do and do what it needs to do.

There are so many areas that can be addressed for automating that it would take dozens of pages for them all to be discussed. This blog was intended to give you a start on what you need to do if you truly want to ensure total CIP compliance.

James Holler is founder of Abidance Consulting.

With more and more emphasis being placed on the CIP requirements, some NERC registered entities may be tempted to “relax” and decide that  they are not deemed classified as a Critical Asset. A word of advice: Take a deep breath and think carefully. If you think that it is just too hard to get compliant and the easiest solution is to just declare that you are not critical, you are very mistaken.

Several attorneys that work in the NERC space have commented that those who deliberately or appear to have otherwise deemed themselves to not be critical simply to avoid having to comply, will most likely face egregious fines from NERC…and we’re talking in the millions of dollars. On the same side of this coin, if your CIP program is so weak that you are judged to not be in compliance, you could suffer much larger financial losses, in addition to any fines, if you are the victim of a cyber attack or an interruption due to your failure to comply.

NERC and FERC have made it very clear from the beginning that they are serious about having registered entities comply with their rules and regulations. Since June 2007, NERC fines total more than $35 million and the FERC fines are almost $120 million.

If your organization doesn’t have the appropriate staff on hand to get the job done, then you have two valid options: 1) hire the appropriate staff, or  2) hire a competent consulting firm. Don’t think that you, as a registered entity, are going to be able to “slip one by” the auditor. The eight RRO’s as well as NERC and FERC have hired on some very skilled cyber security auditors that know what to look for and also where to look. These auditors are very good and the only way to “beat them” is to have a great CIP compliance program in place.

You may ask, “how long does it take to get compliant…and for how much”. This is not an easy question to answer as there are numerous variables that determine the answers to these questions. A few variables are…what is your current state of readiness; have you leveraged from other compliance effort areas such as Sarbanes-Oxley, HIPPA and PCI; have you tested your current state of readiness against a mock audit? There are dozens of factors that must be considered before you can even guess at the timeframe and associated costs. One thing is for sure though…it will cost you a lot less money to get compliant than it will for you to “keep your head in the sand”.

James Holler is founder of Abidance Consulting.

Cloud computing represents a major change in how you store information and run applications. Instead of hosting applications and data on an individual server, everything is hosted in the “cloud”—a collection of computers and servers accessed via the Internet.

This type of Web-based computing frees you from the autocracy of single-server computing and opens up new avenues for group collaboration. But as attractive as all that sounds, cloud computing isn’t for everyone. This blog will take an honest look at the pro’s and con’s of this type of solution and how the average end user can benefit from cloud computing.

Reduced Software Costs – Instead of purchasing expensive software applications, you can get most of what you need for a pittance compared to the $200k+ you will spend buying Documentum or SharePoint. Yes, SharePoint is virtually free…but the programming and maintenance isn’t. This alone may be justification for switching to cloud applications.

Improved Document Format Compatibility – You don’t have to worry about the documents you create on your machine being compatible with other users’ applications or operating systems. In a typical environment where Word 2007 documents can’t be opened on a computer running Word 2003, all documents created by Web-based applications can be read by any other user accessing that application. There are no format incompatibilities when everyone is sharing documents and applications in the cloud.

Unlimited Storage Capacity – Cloud computing offers almost limitless storage. Your computer’s current 200 gigabyte hard drive is peanuts compared to the millions of terabytes available in the cloud. Whatever you need to store, you can.

Increased Data Reliability & Security – Unlike server or desktop computing, in which a hard disk crash can destroy all your valuable data, a computer crashing in the cloud won’t affect the storage of your data. That also means that if your computer or server crashes, all your data is still out there in the cloud, still accessible – there is no “wait time” for a recovery tape to be loaded. Many cloud providers offer military grade encryption…far more secure than anything your organization could hope to provide.

Anywhere, Anytime Access – The ultimate advantage to cloud computing is that you’re no longer dependent on a single computer or network. Change computers, and your existing data and documents follow you through the cloud. Move to a portable device, and your applications and documents are still available. There’s no need to buy a special version of a program for a particular device, or to save your document in a device-specific format. Your docs and their apps are the same no matter what computer or other device you’re using…that goes for Apple computers as well.

Now…just to be fair and not to sound too biased towards cloud computing, there are some pitfalls.
However, I do believe that after you have seen the advantages and disadvantages, you to will decide that cloud computing is still the best way to go. OK…here are some of the pitfalls.

Requires Internet Connection – Cloud computing is impossible if you can’t connect to the Internet. Since you use the Internet to connect to both your applications and documents, if you don’t have an Internet connection you can’t access anything, even your own documents.

May Be Slower – Even on a fast connection, cloud-based applications can sometimes be slower than accessing a similar program on your desktop or server. The one solution to this issue is to “check out” the document. When a user is done working on the document, they can “check in” the document.

So, who are the users that are best suited for cloud computing? Given the pros and cons of cloud computing, I think that the following types of users benefit most from switching to cloud-based solutions and applications:

Collaborators – If you collaborate with other people on group projects, you’re an ideal candidate for cloud computing. The ability to share and edit documents in real time between multiple users is one of the primary benefits of Web-based applications; it makes collaborating easy.

Users With A Need For Total Security – Cloud computing, when properly configured, is one of the most secure environments known today. Many outsourced cloud solutions provide a total package that includes not only all the storage space you will ever need, but also security that would make the Pentagon jealous for and a maintenance program that is worry free. You will save large amounts of money, time and resources by not having to lay out big bucks for the latest versions of Documentum or maintenance programs for SharePoint – both of which have very limited security…if any at all.

Users With Regulatory Compliance Needs – When you are required to comply with NERC, FERC, CFATS or other compliance measurements, there are many areas that you must address. You could hire a high-priced consulting firm with almost no industry experience or pile more work on your already thinly stretched internal resources and purchase a fleet of new servers and desktops, or you could utilize lower-cost cloud computing instead. The other main advantage for those who have to adhere to compliance requirements is that the cloud acts as your back-up site for Disaster Recovery. Abandon that outdated technology and use a less-demanding, low maintenance, fully secured and hosted cloud instead. In the old days (in computer speak, that is last year), the only solution to increased needs was to purchase more powerful hardware and hire overpriced consultants that didn’t know your industry.

Bottom-line: With cloud computing, the solution is in the cloud—which saves you resources, time and money.

James Holler is founder of Abidance Consulting.

In less than a year the sweeping changes to the NERC CIP requirements will become effective. The changes will require that all registered facilities be considered, to some degree, a critical asset. There are going to be three levels of criticality when it comes to CIP – High, Medium & Low. According to NERC, the process and criteria currently being used today for identifying critical assets in the electric system are inadequate.  For example, the current system labels less than 5% of the existing generation facilities around the country to be critical assets, so NERC has identified a new approach in the new CIP-010-1 standard.

The scoping process in the existing CIP-002 standard calls for identification of critical bulk electric system assets, then the associated critical cyber assets.  In CIP-010, there are no “out of scope” bulk electric system assets; instead a categorized list of those assets and their related cyber systems is required.

Framework
NERC has decided to use the NIST 800-53 framework when they are developing the CIP requirements from now on. The National Institute of Standards and Technology (NIST) is the U.S. Government’s defacto standard for Information Technology Security. You can download a full copy here. NIST provides standards and technology to protect information systems against threats to the confidentiality of information, integrity of information and processes, and availability of information and services in order to build trust and confidence in Information Technology systems.

The NIST framework:

• Provides a specification for minimum security requirements for information systems included in the CIP requirements using a standardized, risk-based approach.
• Defines minimum information security requirements (management, operational, and technical security controls) for information and information systems in each such category that are included in the CIP requirements.
• Identifies methods for assessing effectiveness of the CIP security requirements.
• Brings the security planning process up to date with key standards and guidelines developed by your security team using the NIST framework.
• Provides your security team with assistance in determining what needs to be done and in chronological order.
• Evaluates security policies and technologies developed by your security team.

Major Changes
Be warned, there are many major changes coming. One of the most interesting is that CIP-002-2 through CIP-009-2 will be removed and replaced with CIP-010-1 and CIP-011-1. CIP-011-1 is almost 30 pages and combines CIP-003-2 through CIP-009-2 into a single requirement and includes new requirements as well. The following is a list of some of the major changes on the horizon:

• Every requirement will be auditable and not just addressable. This means that you must complete all required tasks in the CIP requirements as they will pertain to you and not be a nice-to-have or addressable.
• There is currently a 3-year review/audit cycle set up and because the BES does not change too much or too often that cycle is going to be shortened to be between 12 months and 24 months.
• A new feature in CIP-011 is how the requirements are presented, which is based on applicability/impact on the reliable operation of the BES.  There are several subject areas identified in CIP-011, including: security governance and policy; personnel training, awareness, and risk assessment; physical security; electronic access control; etc.
• Each requirement has several characteristics identified, and each requirement is assigned to one of the subject areas.
• The need for more than paper evidence of compliance has lead to actual need to demonstrate compliance in the updated version of the CIP requirements. For example, current requirements call for paper demonstration rather than allow for actual demonstration of the protection system; the latter improves security and therefore an entity will have to demonstrate their compliance rather than state it.

There are many, many other updates, improvements and additions to the upcoming CIP requirements known as Version 4. It is my opinion that a registered entity may want to begin preparing now because the requirements may prove to be difficult to handle.

James Holler is founder of Abidance Consulting.

Every NERC registered entity eventually needs a consultant to assist them with their NERC Compliance programs, but not every registered entity is up to the task. When is it time to call for help? And how do you know which consultant is right for you?

As a NERC compliance consultant myself, I’ve seen the good, the bad and the ugly when it comes to some of my colleagues out there. I’ve also seen situations where an otherwise good consultant was simply the wrong fit for a client’s specific situation. My hope here is to help you learn from what I’ve observed over the years to help you first determine if you need a consultant, and then to pick the best one for you.

First, be honest with yourself and consider your skills. If you have a hard time defining what “annual” means or how to create and implement an Internal Compliance Program, then developing a full blown NERC Compliance program on your own may not be for you. Even if you consider yourself knowledgeable, and know the difference between a critical asset and a non-critical asset, you may be getting in over your head.

Time is another major consideration. Do you have enough free time to get the job done correctly? There is nothing worse than completing your NERC Compliance program only to find that all of your hard work was wrong when you are audited. If you are not confident you have the skill, the tools or the time you might need to call in a professional consultant. Finding the right person for your project is not as easy as grabbing the yellow pages and making a call; it takes some homework and reference checking.

There are contractors as well as professional consulting firms and then there is the “guy who has heard about NERC” and is now calling himself/herself a NERC Compliance expert. A contractor is someone who you can use for a quick one or two week project, but for the bigger projects that require the skills of a variety of people, you need a professional consulting firm. A contractor will handle all aspects of the small job — from scheduling his/her time to submitting timesheets and invoices to you. The registered entity will sign a contract with this contractor and make payments only to him/her. He/she will handle all the details of the job and will be the only person the registered entity deals with. Any concerns should be directed to him/her and he/she will hopefully do what is required.

When you have a full blown project that needs to be completed, it’s probably time to bring in the professional consulting firm. These firms will carry millions of dollars worth of liability insurance, have numerous staff members that each specialize in different areas and, in some cases, can take over your entire NERC Compliance program from you, thus allowing you to go back to producing power, etc. Professional consulting firms will have multiple services and products that are geared towards your compliance efforts. A professional consulting form will have account managers, project managers etc., so that at each step of your project you can be assured that your best interests are being considered.

You wouldn’t go to a podiatrist for a root canal, and you wouldn’t want to hire a roofer to hang your wall paper. You need to find the right consulting firm for your NERC Compliance project.

The best way to find someone is to ask other organizations like yours for a good recommendation. Don’t simply rely on the smooth talker who uses big words like “quantitative” and other nonsensical language that is meant to show you they know a big word. Having an ad in the back of the monthly magazine doesn’t equate to having any skills or ethics. Those ads are often inexpensive and easy to get.

If a reference or commendation isn’t available, take the time to interview a handful (2-3) of consulting firms and ask them for 5 to 7 references. You may also want to call the local Better Business Bureau to see if any complaints have been filed against any of them. Get bids from the ones you felt comfortable talking with and then go with your gut. Did they return your calls promptly? Did they show up on time for the meeting? Did they present well?

Another important tip: Always get “not to exceed” quotes from the consulting firms and don’t automatically jump on the lowest bid. The cheapest firm with one or two staff could be very skilled and knowledgeable and just right for the job. Then again, they could be uninsured and just looking for quick money before they move on to the next unsuspecting client. Once you do decide on a firm, ask others in your area or friends in the power industry if they have ever heard of the consulting firm(s) that you are considering.

Ask for proof of insurance up front when getting bids. There is both liability insurance and workman’s compensation insurance. Any excuse for not having the paperwork handy is a sign they are not insured. If a contractor gets injured at your site and breaks every bone in his body, and is not insured, you are liable and could end up paying the big medical bills.

Once you feel comfortable with a consulting firm, get everything written up in a contract. Make sure that the start and finish dates are clearly stated and all materials being used are listed in detail. This is where a good relationship can go bad. A “facility ratings methodology” is not the same as a “facility ratings methodology that has been audited against the standard.” Set all expectations in writing. Any changes discussed while the project is in progress should also be put in writing (this is called a change order).

Some consulting firms will want to be paid based on the time it takes to do the job and the materials/tools used. It is far too easy for the job to run long when paying by the hour. A better approach is to pay by the job or a “not-to-exceed”. And, never, under any circumstances, pay any amount up front. The best payment structure is to pay on Net 30 terms…unless the consulting firm offers you a 1% to 2% discount for paying early. Be sure this arrangement is clearly outlined in the contract.

When considering a NERC Compliance project, critique your own skill level, check your “tool box” and determine whether or not you have the free time to finish the project in a timely manner. You may find it to be a whole lot easier to hire a professional consulting firm and go back to doing your “real” job instead of doing it yourself.

James Holler is founder of Abidance Consulting.