GRID ARCHITECTURE ARTICLE 5:  How Do We Measure the "Goodness" of Complex Architectures?

Posted to GridIntellect, LLC – A Veteran-owned Company in the Digital Utility Group
image credit: Stuart McCafferty
Stuart McCafferty's picture
Lead Architect, eaaS Siemens Smart Infrastructure CTO

2021 Cleanie Award winner. Siemens Smart Infrastructure CTO Office, technologist, distributed energy expert, researcher, author, and climate change warrior. Genuinely focused on doing good for...

  • Member since 2018
  • 112 items added with 103,827 views
  • Sep 14, 2020

GRID ARCHITECTURE ARTICLE 5:  How Do We Measure the "Goodness" of Complex Architectures?

by Eamonn McCormickStuart McCafferty & David Forfia

"Determining what constitutes "Good Architecture" is a major challenge for complex architectures"

Dr. Jeff Taft creator of the Grid Architecture Methodology

In this fifth article we address the question every architect asks him/herself, “Is my architecture good?”  What does a good architecture look like? These questions are particular difficult to answer in complex architectures where we are dealing with a very large number of variables. Key to this idea is how do we measure the "goodness" of an architecture. Despite all of the literature about architecture, there is actually not much written about how to measure architectures themselves. A typical answer is that architecture contributes to improved business outcomes. Therefore, if the business outcomes are achieved, one can assume that that the architecture was good. This may be true to the extent that a "bad architecture" may doom a business to failure, however it does little to answer how we actually measure an architecture’s “goodness”. 

Measuring Goodness, the DOE Way

This topic was addressed by Dr. Jeff Taft as part of the Grid Architecture methodology developed by the Department of Energy’s (DOE) Pacific Northwest National Laboratories (PNNL) as a framework that the industry can use to solve its most pressing problems. In a nutshell, Dr. Taft proposes that we can evaluate the goodness of an architecture by assessing how well the architecture supports the external behavior, or “qualities”, needed by the stakeholders. This outside-in approach to evaluating an architecture’s goodness makes intuitive sense.  A good architecture must unambiguously serve the needs of the stakeholders as the primary directive, and that is what goodness is all about.

Unfortunately, in highly complex architectures we often do not begin with a clear understanding of this "outside-in" approach and confuse the internal perspective of architectural “properties” with the external perspective of “qualities”. This confusion can lead to serious challenges. For example, if we look at transportation from the perspective of the "customer needs", we end up with solutions that may or may not include car ownership.  Hence, we have serious business issues for car manufacturers whose architectures assume "sale of a car" versus an external stakeholder’s need "to get from point A to point B efficiently". Similar challenges are now emerging in the energy landscape. What kind of qualities should an energy service have? Before we design a "grid" to deliver services, it is critical to recognize customer needs. For example, the types of services needed by an electric vehicle customer may be significantly different than those of a gasoline vehicle customer.  

Another emerging customer stakeholder “quality” that is emerging is around the idea of a utility’s architecture supporting solar photovoltaic (PV) installations behind the meter.  In fact, there are several Environmental, Social, and Governance (ESG) architectural qualities emerging from the European Union that include energy storage, energy efficiency, microgrids, wind generation, electric vehicles, and other carbon neutral/minimizing technologies.  As the US population’s concerns about climate change and becoming a more sustainable nation, these ESG qualities are already being considered by most utilities performing Grid Architecture exercises.  Catering to these new qualities becomes of paramount importance when developing architectures that address these critical customer needs.

Figure 10: EU Taxonomy for Sustainable Financing[1]

The fundamental takeaway is that the goodness of an architecture is measured with respect to how well it meets the external stakeholders needs. At the early phase of an architecture it is very important to get everyone aligned on this idea.  Also, qualities should be independent, and non-overlapping to avoid double counting when actually calculating “goodness” – more about that later. 

In the Grid Architecture methodology, Dr. Taft has defined qualities and properties accordingly:

  • Qualities are architectural classes of requirements from an external point of view
  • Properties are architectural classes of requirements from an internal point of view

Both qualities and properties are classes of functional and non-functional requirements with qualities being the measure of architectural goodness.  For most of us who have practiced architecture for a long time, we have become used to taking an external services view of architecture.  However, clearly differentiating qualities from properties is a very important innovation, because so often we as architects still mix up the qualities with the properties as we look at services. We often jump to conclusions around "ilities" of the systems needed to support the "services" without questioning if they really meet stakeholders’ needs and whether we really understand the qualities.

In traditional architecture we are used to mapping requirements to functions to components.  This works reasonably well for solution architectures but does not work as well for highly complex architectures, such as Grid Architectures.  The problem is simply the vast number of functions and we miss the big structural concepts that underpin complex architectures.  By building classes of requirements separated as qualities and properties, we are in a better shape to map needs to the various architectural structures and their components.

A key element of any architecture is to determine if it is a good fit for purpose. As we are now looking at the fundamentals of energy grids, we need to be careful not to make poor assumptions about what qualities are required for whom in the future.

In 2015 the DOE did some thinking about what qualities would make the grid "good by 2030"

  • Minimal environmental footprint
  • Affordability
  • Flexibility
  • Extensibility
  • Interoperability
  • Optionality
  • Safety
  • Scalability
  • Reliability

It is difficult to calculate whether an architecture meets all these "ilities" qualities rather well or rather badly. The "ilities" are often overloaded, hard to measure, and vague.  It therefore makes sense to spend time to identify the different external qualities and the internal properties needed to support them before jumping into architecting components. 

Mapping External Qualities to Internal Properties

Figure 11:  Example Mapping Exercise of Qualities/Properties/Elements[2]

In the Grid Architecture methodology grid Qualities are mapped to Properties and Components/Structures are mapped to Properties – in that order.  So, in the above graphic, the mapping process starts from the right-hand side System Qualities to the center Key Properties.  The Key Components and Structures on the left-hand side are then mapped to the center Key Properties.  This visualization allows the Grid Architecture team and its stakeholders to recognize how the structures and components meet internal property requirements and the external system quality requirements.  In other words, it allows us to see how well elements are actually contributing to outcomes that are meaningful to customers. Surprisingly, we may find that some elements do not contribute strongly to the outcomes customers and stakeholders most care about.  In this case, we should question whether this is the right set of elements to support the outcomes we are looking to achieve.

Of course, this is a simplification of how it works in practice. Elements themselves are related to each other and we must "iteratively" explore the relations between qualities, properties, and elements.  But the bottom line is that we must ensure we are sufficiently focused on how the "structural elements" of the architecture support both qualities (external needs) and properties (internal needs).   

Optimization of the architecture does not imply perfection, but rather minimizing the amount of work necessary to meet the internal properties and external qualities.  In other words, what are the minimal number of structural elements necessary to meet the architectural properties and qualities?  We can also think about what elements contribute to the qualities and how valuable each quality is.  An interesting exercise is to relax the qualities to determine if this can lead to a large decrease in complexity...

PNNL is working with the GridWise Architecture Council (GWAC) and DOE to further advance this important topic.

[1] Taxonomy: Final report of the Technical Expert Group on Sustainable Finance, Financing a Sustainable European Economy Technical Report, EU Technical Expert Group on Sustainable Finance, March 2020

[2] PNNL, Grid Architecture Training, Dr. Jeff Taft, 2019


No discussions yet. Start a discussion below.

Stuart McCafferty's picture
Thank Stuart for the Post!
Energy Central contributors share their experience and insights for the benefit of other Members (like you). Please show them your appreciation by leaving a comment, 'liking' this post, or following this Member.
More posts from this member

Get Published - Build a Following

The Energy Central Power Industry Network is based on one core idea - power industry professionals helping each other and advancing the industry by sharing and learning from each other.

If you have an experience or insight to share or have learned something from a conference or seminar, your peers and colleagues on Energy Central want to hear about it. It's also easy to share a link to an article you've liked or an industry resource that you think would be helpful.

                 Learn more about posting on Energy Central »