Chapter 2. Functions of a Lessons Learned Capability
Lessons learned (LL) programs will vary according to the needs of the organization; however, most will have at least six functions as shown in Figure 2-1. Each of these functions is explained in greater detail in this chapter.
Figure 2-1. LL functions
The first function of any LL capability or program is the collection of information relevant to an issue that someone has determined requires analysis. In an LL program, the issues are usually topics the organization realizes are causing concern or problems, and the organization wants to determine a better way to do business or enhance performance. Historically, most organizations are passive when it comes to reporting problems and potential solutions shared with other like organizations so those other organizations do not encounter the same difficulties. Normally, if someone does not make the effort to "pull" these issues from the organization, do not expect them to "push" them to you. Many reasons exist for this, such as operational pace, shortages of manpower, time constraints, a lack of understanding the importance of sharing information, or no process in place that facilitates the sharing. Ultimately, the goal of any collection effort is to gather enough information to have informed analysis that resolves the issue so other organizations can benefit from the experiences of those who have gone before them.
We do not collect lessons to evaluate an organization; rather, we collect lessons to help organizations improve. If perception of the LL program is as an evaluator, an inspector, or an agency of internal review, no one will be willing to share problems and corrective actions. A policy of nonattribution may be appropriate. Make every effort to avoid personally naming individuals; instead, use duty positions or work positions. Try to avoid naming specific units or offices if possible. For example, if you were collecting on the 1st Battalion, 502nd Aviation Regiment, instead of saying 1-502 Aviation, simply say an aviation battalion. In other words, be generic when it comes to sources but specific when it comes to issues. To get the desired access and have credibility with your organizations, the LL program must be seen as trustworthy, unbiased, and able to guard sources when asked to do so. That said, information gathered from a collection should not be watered down or altered. Collect the facts and only the facts, and try to avoid opinions.
Finally, there is a tendency in any LL program to focus on what is not working well (the negative). A good LL program should also collect information on what is working well (the positive). "Best practices" are what we call positive observations. Certainly, the tendency is to focus on the problem areas, but you may find organizations that have mastered a certain task and now can share those best practices with others. As a rule of thumb, expect that 80 percent of what you collect may be negative (in an attempt to improve); however, 20 percent should be positive best practices.
Opportunities to collect information on issues conform to the mission of the organization the LL program is supporting. For example, a military organization may have the responsibility to protect our nation from foreign adversaries but also provide support to civil operations in times of national emergency. When you analyze that mission, there are many collection opportunities that exist that have the ability to provide vital observations and lessons on how to best conduct those operations in the future. Although the participants should be the first line of observation reporting, often it becomes necessary for the LL organization to be proactive in the collection process. The following is a potential list of collection opportunities:
As you can see from this list, there are several opportunities to gather information. To get the maximum benefit from these opportunities and match them to the available resources requires a long-range collection schedule be developed that overlaps the various opportunities and phases them in over time so the workload is manageable. Typically, this schedule runs for one year and makes an effort to lock in dates and locations for each collection effort. In the example above, the occurrence of a natural disaster is difficult if not impossible to plan for; therefore, the LL program must have the capability to initiate impromptu collections on very short notice. Independent LL organizations may need a collection schedule; however, organizations with smaller, internal LL programs may not.
Determining critical issues
One of the greatest challenges of the collection phase is determining what exactly are the issues that require a collection effort be analyzed. There are several ways to accomplish this. Often issues are "command directed." A higher headquarters, agency, or organizational leader directs you to collect on a certain issue of interest to them. This makes life simple and usually allows you to avoid a deliberate process whereby the LL program must decide what issues are most important. Sometimes a "shotgun" approach is effective. This approach understands that in every complex operation there are always issues that require work, and if the LL program is working, the issues will certainly be discovered. The disadvantage with this approach is that you do not focus and you have a tendency to look at everything, which dilutes your efforts.
Another method might be the feedback gained from multiple units or organizations that have expertise performing a certain function that is not working properly. When numerous experts in the field are all reporting the same issue, it is probably critical. For example, after the initial invasion of Iraq in 2003, the U.S. Army experienced problems defeating the rocket-propelled grenade that was destroying the new Stryker combat vehicle. Because of this immediate feedback from commanders in the field, the Army devised a defensive shield that could be constructed around the vehicle to detonate the missiles prematurely, rendering them ineffective.
Finally, a more analytical and deliberate approach would require some degree of analysis at home station to determine the issues that are most important to your leadership. A deliberate approach focuses your time and resources but requires a decision process to be developed to determine what is critical. One way to do this is to study past operations, reports, articles, and briefings and begin to compile a list of issues that seem to reoccur and cause difficulty. You can weight each issue with its degree of importance, assess risk, rank order the issues, prioritize resources, and make a recommendation to senior leadership on which issues are most important for the purpose of collection. Each collection effort will more than likely uncover new issues that can be added to the existing list. Once again, you will need to go through a process to determine which issues are most critical for future collection efforts.
The collection plan
Once you have determined the critical issues, you will need to develop a collection plan to guide the collection. A collection plan can be as simple as a list of questions you desire to ask the organization and its subject matter experts (SMEs) or as detailed as the following:
However, the common threads in all collection plans are the questions. You never answer a good question with a yes or no. The question should be worded in such a way that the person answering the question must give a full account of the process the organization is using. Follow these examples:
WRONG: Is your unit performing route clearance operations?
As stated previously, you must avoid the tendency to collect on too many issues. More is not necessarily better. As a general rule but based on the size of your collection team, more than 10 issues for any one collection effort gets cumbersome. If a collection team can deploy with six to 12 lead questions already prepared for each issue, that normally gives you the degree of fidelity you need to get good results. Additionally, the collection plan is only a plan. Once you begin to ask questions to the interviewees, there is nothing wrong with developing new questions "on-the-fly" that better address the issues based on the information you are gaining from the unit's SMEs. Do not get locked into the plan. It is only a start-point to get you moving in the right direction.
Finally, there is another collection plan technique you may or may not elect to use. It involves the use of a hypothesis statement for each issue. The hypothesis statement is a statement that defines what you want to confirm or deny about the issue. For example:
Hypothesis: Brigade combat teams continue to have difficulty clearing routes due to the enemy's use of improvised explosive devices (IEDs).
In this example, there is probably some anecdotal information that indicates units are experiencing problems with this enemy tactic. Focusing on the collection plan will confirm or deny this issue. The use of a hypothesis statement is sometimes more beneficial in focusing your collection efforts when you are trying to confirm that a corrective has been applied and a change of behavior is anticipated for the better. For example:
Hypothesis: The implementation of counter IED-defeat mechanisms has greatly reduced the threat of IEDs along routes traveled by brigade combat teams.
Figure 2-2 gives a sample collection plan format for briefing purposes on one issue. Again, the plan can be as detailed or as simple as the observer desires. The sample shows the issue and hypothesis, lists lead or key questions to ask, and describes the interviewees or individuals who will be asked the questions. Appendix A gives an example of a very detailed collection plan, and Appendix B gives an example of interview techniques.
Direct and indirect collections
Now that you have a collection plan, you must begin to decide (if you have not already done so) how you intend to conduct the collection. An LL program will normally use two different types of collections, both accomplishing the same tasks but each using different resources. The direct (sometimes called formal) collection effort always involves detailed planning and is generally a top-down driven process. The indirect (sometimes called informal) collection effort uses much less planning and generally fewer resources, but is typically a bottom-up process.
An example of a direct collection would be a team of SMEs assembled and trained on the collection process and how to build a collection plan, focused on specific issues generated by a higher authority, and deploying overseas to complete a 30-day collection on a specific organization. An example of an indirect collection could be as simple as a Soldier in a military unit submitting an observation he wrote to his headquarters, which forwards it up the chain of command, eventually getting to the LL organization. This is sometimes called an unsolicited observation. It may or may not be tied to an existing issue, but it is information that someone deems critical enough to take the time and effort to submit in writing from a lower echelon. Although it is always challenging to get organizations to submit unsolicited observations, they are typically some of the best lessons an LL organization receives.
Figure 2-2. Example collection plan briefing
Another informal collection technique may be simple research. If a qualified analyst or researcher conducts extensive research on an issue and finds the right information and lessons, it may prevent the requirement for a direct collection effort, saving time and resources. In either case, the researcher or observer in each instance should have some degree of expertise in the area he or she is exploring, hopefully an SME who can provide valuable insights into the operation.
The only reason for understanding the distinction between direct and indirect collections is to realize there are many ways to get LL. Some ways are more challenging than others. It may sound difficult to put together a direct collection team, but it is sometimes harder to get feedback from the field. Anyone involved with LL programs for any amount of time understands this. If timely feedback was so prevalent, there would be little need to deploy direct teams.
Another technique for collection could at times be direct or indirect. It concerns the use of embedded teams or liaison officers. The U.S. Army has effectively used this concept during the Iraq and Afghanistan wars. They brought Reserve officers on active duty, formally trained them on collection techniques, helped them develop a collection plan, and then deployed them into theater for periods of six months to one year. Each officer was embedded with a unit or a headquarters. They provided vital and timely information on unit operations; tactics, techniques, and procedures; and lessons learned. This effective "push" system was probably a direct collection due to the planning and resources involved. Other techniques might involve the use of survey instruments, or the positioning of liaison officers with various organizations that have their own LL programs, to maximizing the sharing of lessons across agency boundaries.
The military uses a very instructive process as outlined in Appendix C called the after action review/report (AAR). You would think that this is an indirect process but one that requires a great deal of effort and time to execute. This process is one that produces some of the best and most timely lessons.
Although the U.S. Army developed these concepts, with some modification they could be adapted for interagency, intergovernmental, or civilian uses.
Since the direct collection is probably the most complex to execute, it is beneficial to understand the process to execute this task. Figure 2-3 depicts an example of what a direct collection team may look like. This example shows four observers/SMEs, but the team could be as many as 12 members if the issues support that size. More than 12 are cumbersome, especially if you are traveling overseas.
Figure 2-3. Generic direct collection team composition
The team chief has the responsibility to lead the team, validate the collection plan, review observations drafted by observers, meet regularly with the team to review progress, assist in writing the observation report, and conduct any briefings required to introduce the team or provide an out-briefing when the team departs the organization it is observing. The team chief should be of sufficient position or rank to "open doors" if the requirement exists. In the military, the team chief is usually a colonel.
The operations officer is the team chief's deputy and his "right-hand man." He is usually a full-time member of the LL organization with prior experience supporting collection teams. He performs administrative tasks such as equipping the team, making travel arrangements, and scheduling interviews, and helps the team chief review observations for correctness.
Observers/SMEs make up the remainder of the collection team. Observers have the responsibility to be objective, do unbiased reporting, have no personal agendas, and be good writers, since they are responsible for drafting all of their observations for the observation report in the formats described below. Besides the use of a survey tool, the primary way to get information as an observer is through an oral interview. The observer should be an SME as well as the person he is interviewing. Information on how to conduct an oral interview is in Appendix B.
Training for any collection team is important. Normally, three or four days of training before the team departs is sufficient. Subjects taught in this "predeployment" workshop are observer duties; security training; collection plan development; and any training required on issued equipment such as laptop computers, cameras, satellite phones, and voice recorders. However, the main purpose of the predeployment workshop is to finalize the collection plan to the team chief's liking. It is also important to conduct a "post-deployment" workshop. The primary purpose of this workshop is to draft the observation report as described below. A secondary purpose is to return issued equipment, finalize travel arrangements, complete a security debriefing to ensure there is no compromise of classified information, and give a commander's or director's briefing.
The result of a direct collection effort should be a well-written observation report. Although modification may be necessary to correspond to any situation or desired format, the following sections are included:
You can use several different formats to write an observation. One of the easiest to use is the issue-discussion-recommendation or observation-discussion-recommendation format. Another format the U.S. Army uses is the following:
The joint community uses the following observation format:
NATO uses the following observation format:
NATO also has a format for a best practice:
The key to any format is that you thoroughly discuss the observation using facts and examples while trying to avoid opinions, highlighting any lessons and making recommendations to fix the problem if you have the expertise to do so. When dealing with LL, do not mistakenly write a truism instead. A truism is something we know to be true at all times. At one point, it may have been an LL (a long time ago), but over time, it becomes enduring. For example, IEDs are a major killer on the battlefield. Years ago, at the start of the war on terrorism, this was a lesson learned; today, it is a truism. The LL, in this example, would be ways to defeat IEDs.
A good length for a written observation is one to two pages. Another thing to keep in mind is that the "shelf life" for an observation is about six months. This means that anyone who attempts to use the observation to justify an effort after about six months needs to confirm with the source that the observation is still valid. For example, maybe the source organization determined the observation will not be corrected for whatever reason, or maybe the observation was corrected and it is no longer an issue. Using an old observation or observation report that is no longer valid to substantiate or justify a project without making sure it is still an accurate assessment of the issue will show that you have not done thorough research.
For this reason, some LL organizations prefer to keep raw observations as internal reports, disseminated only to those who have a need for the information. One technique to use is putting a statement in the front of the report that "warns" the user after a certain period of time to make sure they verify the status of each observation before citing them in another project. If you have the time and energy, a very good procedure is to update the status of each observation once any new information develops concerning its resolution. This requires a conscientious staff and considerable time and effort to monitor and track the actions associated with each observation, most of which may not directly involve the LL organization. The issues- resolution process, which will be discussed later, can provide information to support this effort.
Make every effort to keep the observation report classified at the lowest level possible. That said, it is the responsibility of the observer to ask the interviewee the sensitivity of the material he is receiving. If there is any doubt, always have the unit intelligence officer or facility security specialist confirm the classification of the information in question. Be especially careful with photographs taken or briefing slides with embedded pictures. Organizations that deal with classified information on a daily basis are sometimes sloppy in their handling and marking of documents.
Finally, out of courtesy, it is appropriate to let the organizations interviewed review a final draft of the observation report. In some instances, they may make recommendations that clarify what was given in the interviews. Typically, they should have at least two to three weeks to review the observation, a time you need to factor into your planning sequence.
What you must avoid is the tendency for the reviewing organization to "correct the report." Any LL program has the responsibility to present only the facts. One way to compromise on points of tension is to include the unit's opinion as an appendix. This should keep both sides happy.
Figure 2-4 highlights the LL process and where the collect function fits.
Figure 2-4. The LL process with the collect function
Analysis is a process used to thoroughly understand areas of activity identified to have potential for improvement. You can accomplish this action at the end of the event or after the active collection has been completed; however, a certain amount of analysis may be done throughout the LL process. Transforming the raw data into actionable recommendations requires a systematic process to examine the information that has already been collected and understand why or what contributed to the need for improvement. The level of analysis may be determined by the expertise or resources available or by time limitations in developing a final product. In some cases, it may be more important to conduct a surface analysis and expedite the results back to the user so they can start to take corrective action, where as with other observations, the complexity of the issue or resolution may necessitate a more detailed analysis and explanation. In either case, raw observations may change in context, content, conclusion, and applicability during analysis.
Validation of the observation
During this part of the analysis, you start to organize the data you have collected, ensure you have explored every possible resource, and agree on the direction and method the analysis should follow.
Frequency of occurrences. Review previous observation reports, AARs, LL databases, and other reports to determine the frequency and conditions in which the observations have occurred. Is there a solution on record? An effort should be made to determine if the issue has been previously identified and a solution initiated but not yet implemented.
Understand the objective. It is important to understand where to focus your analysis. While conducting the analysis, it is easy to identify additional issues that may cause you to stray away from the intended objective or the customer's needs. Consider capturing these issues and setting them aside for later collection efforts.
Review findings with the host organization or other stakeholders. If not already accomplished during the collection phase, review the observations with the unit or organization where it was collected to determine if there were any unusual or contributing conditions for the issue. Do not provide recommendations at this point, since they may be based on incomplete analysis.
Analysis of observed data
This is the step where the analysts start to analyze, brainstorm, and dissect the information collected. Use different perspectives when looking at the data to fully understand the issues, examine each piece to see if other issues exist, and start to develop the full story. It is critical to discover not only what happened but why it happened.
During this phase you start to organize the results of the analysis and determine appropriate recommendations. If you do the collection and analysis correctly, the recommendations should be intuitive. The recommendations should say what needs to be done and not just what effect needs to be achieved.
The end state of the analysis process should be a well-defined list of recommendations or potential corrective actions with sufficient detail to be shared, achieved, and entered into the issues-resolution process. If done correctly, the analysis will help determine the action plan and resources required to implement the corrective actions.
Figure 2-5 highlights the LL process and where the analyze function fits.
Figure 2-5. The LL process with the analyze function
LL programs must have the ability to share and disseminate information to be effective. Besides the ability to share information, the LL program must be able to determine what information is important or urgent and how rapidly it must be passed to other organizations that could benefit from the knowledge. The LL program must have a process and medium to do this. You can share through several means, such as printed "hard copies," electronic forms like e-mail and messaging, collaborative forums, and websites. The process should support the capabilities of the medium. Additionally, you should have the ability to handle both classified and unclassified material.
Resourcing of the LL program will usually determine what dissemination means are used; however, it helps if the resourcing conforms to the information required by the target audience to successfully execute their missions. For example, printing professional, hard copy documents can be expensive and may exceed the budget limits of the LL organization, but some deployed units who are the target audience may not have access to computers, printers, or copiers in a field environment and need hard copy materials to do their mission effectively. In the future, social networking sites similar to Facebook, Twitter, and LinkedIn may provide additional capability to share lessons if they can be adapted to the requirements of an existing LL program. Finally, it is imperative that all LL organizations within a common community be linked to facilitate the flow and rapid exchange of information.
Prioritization of Information
The key to information or LL dissemination is a rapid sharing process. This requires an ability to rapidly analyze information from collections, determine relevancy and timeliness, and gain permission from leadership to share. The challenge in this process is the quicker you need to get the information out, the more risk you assume in conducting a thorough analysis to make sure you are drawing the correct lessons.
An example of one way to construct a rapid sharing process is depicted in Figure 2-6. In this example, the LL program has created timelines for sharing that are tied to the urgency of the information and a medium to disseminate that information. The terms immediate, urgent, and routine would need to be specifically defined to meet the goals of the supported organizations and their mission. For example, in the military, an "immediate" requirement to share a particular lesson, if not shared, may result in the injury or death of a Soldier. However, once rapidly shared, information should continue through the analysis process and eventually be formally vetted, archived, and become a part of the issues-resolution process, if it rises to that level of importance.
You can publish lessons of less priority as articles, reports, bulletins, and so forth. The U.S. Army has a medium where any Soldier in an operation can write an article that can be posted on the LL program website for anyone to read. It is called "News From the Front." This is an excellent way to post timely, thought-provoking pieces that discuss best practices; lessons learned; and tactics, techniques, and procedures. An observation report, discussed in the Collect section, requires a longer lead time to produce because it is usually part of a direct collection effort involving a deliberate planning process. Finally, the production of a newsletter, handbook, or other periodical that may or may not go to a print plant for production usually requires the greatest resources and the longest lead times to produce. They may require anywhere from three to six months to develop and print; however, they additionally serve as excellent historical documents that may be very beneficial in supporting future operations.
Figure 2-6. Example rapid sharing process
Publications and automation
As previously mentioned, the production of publications and the ability to disseminate information may be a consideration for any LL program. If you cannot share the information you collect, it is virtually useless. Some of the major functions of a publications capability may be the following:
Some examples of categories of potential publications based on subject matter, size, and frequency are:
Before publishing, one challenge to sharing information is the degree to which the LL program wants to edit material. There will be times when the editorial process should be eliminated or modified to facilitate the expeditious release of information critical to the success of the organizations being supported. As the saying goes, "perfection is the enemy of good enough." The editorial process should not become so burdensome that it prevents a document from being published in a timely manner. On the other hand, if time is not an issue, a professional product is always a good option.
Automation obviously enhances the capability to share information. Although some organizations may prefer to retain a paper-based capability, that capability is usually expensive. Others may prefer a purely automated dissemination system using electronic distribution techniques such as e-mails, websites, and collaborative sites. Some organizations may prefer both capabilities, paper and electronic. However, making a portable document format (PDF) version of a document allows for easy document exchange and is low cost once the initial investment in automation is made.
You must give consideration to what extent the LL program wants to handle classified information. However, if your goal is widest dissemination, it is best to avoid over classification. Some LL programs have the ability to create databases for cataloging observations and lessons that can be shared with other organizations. Not only can these databases store observations, but also they can archive the source material and reports that support each observation. For example, the joint military community uses a system called the Joint Lessons Learned Information System (JLLIS), and the homeland security community uses a system called Lessons Learned Information Sharing. In any event, the challenge of publishing information and sharing it rapidly reduces by the sophistication and level of automation existing in most organizations and agencies today.
Figure 2-7 highlights the LL process and where the share function fits.
Figure 2-7. The LL process with the analyze function
Why don't people share?
If your LL program can begin to break down the barriers to sharing that are listed above, then you will be on your way to increasing the productivity and performance of the organization. Today, there are organizations and companies that have done just that.
The archive function is a broad term that encompasses several required capabilities that most LL programs desire. The ability to archive information and manage records, both print and electronic, allows access to them at a future date. To accomplish this task, digital repositories (which may be called digital libraries or archives) must be developed to store information, facilitate the historical preservation of information, and allow users to conduct research. Make it easily retrievable and available to any requestor by providing a logical system for storing information. Powerful search engines are required that permit rapid, user-friendly searches. Finally, repositories must have the capability to store and guard both classified and unclassified data.
Databases and websites
Databases and websites are necessary to store and access information. Some examples of what an LL program may desire are:
The LL program can take advantage of this technology to increase its efficiency and ability to access information in a rapid manner. One of the greatest challenges an LL organization will eventually face is the sheer volume of information that will accumulate in its archives. You will need to be constantly planning ahead and determining the best new software to use to upgrade your capabilities and be open to new technologies.
The primary reason to archive information is to have the ability to conduct future research. Research today no longer takes place on library shelves; it takes place electronically using the Internet. Searchable electronic archives will generally be of three types:
Searchable databases provide the researcher easy access to the information required. This capability necessitates the use of highly refined search engines. The use of standardized filing and naming conventions, protocols, and accurately tagging documents with the right metadata facilitates the search. The metadata describes what is in the document that leads you to the material you are trying to find. To ensure documents are properly processed, you may want to establish a quality assurance check. Additionally, the LL organization may require the capability to store printed material, therefore requiring a "vault" or physical repository for storage. The vault is also a good place to store back-up compact discs of all digital material archived electronically.
It is also beneficial to determine criteria for inclusion of information in the repository. Some parameters are as follows:
The archives of the LL organization may be for internal use only. However, if outside agencies are permitted to gain access to the archives, they usually have read-only and download permissions.
Finally, the way information is cataloged or organized must be simple and easy to understand. One trap many organizations fall into is when these procedures are designed by information technology personnel and not by the users. To be simple and easily understood, the user must have input. The ability to quickly find what you are looking for is paramount to any research capability. Simplicity is the key; the fewer keystrokes it takes to find a document the better.
Archived information serves a historical purpose, though the archiving and adding to the repository should take place as soon as possible after the creation of the document or information. Years from now, future generations may desire to study the accomplishments of your organization. They may also find those historical examples applicable to current operations. Archiving material where it can be easily retrieved serves this purpose.
For example, when the U.S. Army first realized it would be involved in counterinsurgency operations in Iraq, it was able to draw on documents archived within its LL program from the Vietnam War. Since these documents all existed before the advent of the computer, over the years great efforts were made to scan this information into the Army archives. Having done this, great time and effort was saved when it became necessary to review the lessons of that war to see if they had applicability for the new conflict, which they did.
A more recent example concerned Hurricane Katrina and its impact on the Gulf Coast. The Center for Army Lessons Learned (CALL) searched the Army LL program archives before the hurricane hit shore, looking for information relating to the 1992 Hurricane Andrew disaster in south Florida. CALL then rapidly shared the lessons contained in those reports with Army units preparing to deploy and support Hurricane Katrina recovery efforts.
The lessons were instrumental in assisting units in their planning efforts. As a result, archiving information for future generations is a much-desired capability within any LL program. It will mean your hard work may get used more than once.
Requests for information (RFIs)
Every LL program should develop a system to answer questions or RFIs submitted to it from individuals outside the organization. Providing timely responses to questions is another way of sharing information. Some examples of this process were explained in the Share section of this chapter. The RFI system should be able to operate in both classified and unclassified communication networks. The system, to operate efficiently, should include an e-mail system, a workflow process, a document management application, a structured query language database, Microsoft Web services, and a Web interface. Once information is located to provide a response to the RFI, it can be sent by using links to the documents with the requested information. While anyone outside your organization should be able to submit an RFI, responses are provided based on the individual's rights as determined by their security classification or clearance. The key to any good RFI system is providing a timely response with some degree of analytical underpinnings, so the requestor gets the information in a manner requiring minimal analysis on his part, thereby making the information instantly usable.
If your LL program handles classified information, it must have the capability to review, store, and archive classified publications. Reviewing documents to determine the correct classification instructions or to ensure that documents are properly marked is a time-consuming procedure, typically requiring a separate security office to perform these duties. It will normally require dedicated personnel specifically trained to execute these tasks. Due to the sheer volume of publications coming from various sources, there can be mistakes made by the originators whereby briefings, reports, documents, etc. are improperly marked, which could lead to a security violation or "spillage" of classified data over an unclassified network.
The LL program must have the capability to perform electronic "key word" searches on each document received to determine if classified information is embedded accidently in a supposedly unclassified document. Additionally, you should require the security team to brief and debrief all direct-collection efforts, so personnel are trained on the procedures to handle, store, and transport classified material. Scrub for classified information, and destroy or properly secure all notes and working papers once the collection is completed. Establish rules for the use of removable media such as thumb drives and compact discs early on to avoid loss or spillage of information not intended for public release. In addition, all laptop computers used by a collection team should be purged of all information once the team returns and is done with their use. Finally, consider other aspects of security, which include operational security to protect the organization's plans and procedures and physical security to protect the facility, equipment, and personnel.
Figure 2-8 highlights the LL process and where the archive function fits.
Figure 2-8. The LL process with the archive function
The most challenging component of any LL program is establishing a process to legitimately resolve issues once analysis is completed and as early and quickly in the process as possible. Issues resolution beyond the organization is challenging for several reasons:
Whether or not the LL program has responsibility for the issues-resolution process, the organization's ability to change behavior by implementing a lesson is ineffective unless you observe that change and a determination made that the lesson is learned. In other words, the corrective actions have enhanced performance. To do this requires a deliberate process to commit resources, make decisions, implement those decisions, and observe the results. If the process is cumbersome and too difficult, expect the results to be less than optimal. If the process takes on too many issues, expect the system to become overwhelmed and collapse on the sheer weight of taskings to agencies that simply cannot handle the additional workload. It is better to resolve your top three issues than to attempt to solve the 70 or 80 observations from the last collection. Understand that the level that developed the issue generally has the highest interest in resolving it.
Not all issues require a formal process to resolve. In the military context, if unit commanders have the capability to correct an issue internally, they should do that. It is possible that during a collection effort the unit or organization becomes aware of an internal problem unknown to the leadership. The first objective is to handle the corrective action at the lowest level possible. The issues that rise to the next level of attention are those the unit or organization are unable to correct internally. Most LL programs will focus on these issues. They are issues that require assistance from sources outside the unit's chain of command. For example, a major modification to a vehicle system would be an issue that requires outside assistance and generally exceeds the ability of a typical military unit to correct.
The issues-resolution process requirements are summarized in Figure 2-9.
Figure 2-9. Issues-resolution process requirements
Senior leader involvement
Senior leader involvement or executive-level participation in the issues- resolution process is the key to success. Without senior-level leadership involvement with the authority to task agencies to work issues and reallocate resources, the process fails. For example, successful issues-resolution steps in the U.S. Army required involvement at the three-star general officer level. This commander had the ability to task his subordinate two-star commanders with responsibility for their respective service branches and combat development processes to drive changes to doctrine, training, education, or materiel. If you cannot fix responsibility at the appropriate level, then the process becomes the end state. The process is not as important as the product. The product, in this case, is a corrective action that when implemented, changes behavior. To develop that product is not an easy task and always requires someone to take ownership at a level able to direct others to complete actions and hold them accountable for their progress.
Determination to commit resources
If you want an effective LL program and an issues-resolution process, you should at some point present the issues gathered and analyzed to a leader who can assist in the prioritization process and make decisions to commit resources to solve problems. There will be few if any corrective actions that will not require the expenditure of some resource, be it time, additional manpower, or money. You will quickly determine you cannot fix everything. Obviously, you can accomplish some things with fewer resources than others. For example, a change to written doctrine or policy typically requires less effort and expenditure of resources than a materiel solution for a piece of equipment or new operating system.
To determine where to place your efforts requires a prioritization process or an ability to rank the issues from most important to least important. This may involve a risk assessment. Normally, risk assessments are subjective, and qualitative analysis becomes the norm. In some instances, a military commander, based on his experiences, professional judgment, and "gut" instincts, can make a good assessment of risk. In other instances, the assessment requires the collective input and wisdom from a group of SMEs or other professionals. For example, the military sometimes uses a "council of colonels" to assist in making recommendations to more senior officers for final decision. Civilian organizations may convene a director-level working group or use a board of directors. Eventually, you reach a consensus or you are directed where to place your efforts based on a combination of risk levels and resource availability.
The key is to focus your efforts on the most important things that need to be fixed, where you can get the greatest "bang for the buck." For example, in a military organization, every effort is made to fix those issues that may have life-or-death consequences; they are considered high-level risks. However, the challenge is sometimes greater in prioritizing issues that do not have that clear distinction. The goal of this requirement is to determine what you have the ability to correct and then begin to develop a plan to do so.
The person in charge with the authority to commit resources needs to pick a lead agency to work the issue. In most instances, there will be supporting agencies that must assist the lead agency. It is rare in today's operational environment that only one agency can fix a problem by itself. Additionally, the LL program should not be the lead agency. The LL program supports the resolution effort by providing guidance and information relating to ongoing or subsequent collection efforts that may inform or assist the lead agency in its work. Typically, the LL program does not have the specific subject matter expertise or the resourcing to resolve major issues of a very important nature that require a formal action plan. Its focus should remain on continued collection efforts, analysis of information, and sharing the lessons with other organizations that may experience the same challenges as the agencies working to correct the issue. As stated previously, the first place an issues-resolution process will fail is by not having senior leadership involved in the process from the start. The second place it will fail is by not assigning responsibility to a single agency to be the lead for a corrective action.
Develop an action plan
Once a determination is made to solve an issue, an action plan should be developed by the lead agency with responsibility to work the issue. You should approve the plan, usually at the level that has authority to assign resources. The action plan summarizes the issue, weighs the risk, outlines a way ahead or timeline for resolution of the issue, and assigns responsibility. The plan can also specifically assign the resources required. It can be as complex or as simple as needed. The example at Figure 2-10 shows a simple "quad" sheet format for briefing purposes. Establish more elaborate and detailed timelines in separate enclosures.
Risk levels are more important in initial decision briefings to determine prioritization. If you use risk levels, you must define what each risk level means. Once those are established and accepted, you can replace them with other necessary information categories, such as required resources/ constraints, essential tasks, or proposed end state. You can use any combination of categories to convey the necessary information and satisfy the desires of the briefing audience, as required. The goal of the action plan is to track the progress and milestones of the lead agency toward resolution of the issue.
Figure 2-10. Example quad sheet action plan
Implement corrective action
Implementation of a corrective action is seldom the responsibility of the LL program. This, again, is one reason why senior leadership must be involved in the process. In some instances, training may be required before implementation. Implementing a corrective action could be a long and deliberate procedure well beyond the ability of any LL program to manage or control, or it could simply be a change to an existing policy or procedure. Like any solution to a problem, there will usually be costs to implement in terms of time, people, and/or money. For example, the U.S. Army categorizes change using seven categories: doctrine, organization, training, materiel, leadership and education, personnel, and facilities (DOTMLPF). However, changing doctrine is generally easier to do than changing an organization design, etc. The LL program exists to facilitate the change process by providing observations and lessons for analysis and consideration for incorporation into an issues-resolutions process whose goal is to enhance organizational performance.
Verify the lesson is learned
Verifying the corrective action or lesson is "learned" once again comes under the oversight of the LL program. It will require an additional collection effort to validate the results. Whether that feedback is gained by a direct or indirect collection effort, the process is not complete until someone determines there is a change in behavior as a result of the corrective actions applied to the problem. The LL program must consider these additional collections as part of its overall responsibilities and plan/budget for them. This requirement to verify the lesson is learned is one reason why the LL program should remain as a "player" and not the "coach" of the issues- resolution process. By doing this, the LL program can remain the "honest broker" within the process.
You can make a case that in the military community a lesson is never really learned because of many factors such as personnel turnover, level of training, and changing operational environment. From the perspective of a doctrinal or materiel solution, this may not be as true. However, what this does point out is the human dimension of military operations. When an organization is centered around people, as is the U.S. Army, the ability to change behavior is an ongoing process. This at times can frustrate an LL program, because it may begin to see the same mistake made over and over again, even after a corrective action was applied. This should not discourage the program or indicate it is ineffective. Any good LL program understands this and adjusts accordingly by continuing to provide the most accurate information possible, so decision makers can decide how they want to prioritize and focus resources to resolve new deficiencies while maintaining an ability to observe the success of implementing past corrective actions. The most important point to remember is that an observation does not become an LL until behavior has changed.
Figure 2-11 below highlights the entire LL process by function.
Figure 2-11. The LL process
There are several ways to determine if the LL program is effective. You can evaluate any LL program by the expenditure of resources against the desired results. It is difficult to determine through quantitative analysis. However, there are some ways to evaluate the effectiveness of an LL program that may validate and justify its need. Assessment of LL effectiveness can be broken into several components: organization behavior, organization or unit performance, and mission effectiveness.
If the training, education, or testing plan/program is not exhibiting the same errors or mistakes identified and addressed by the LL program, it is reasonable to conclude that the LL program has had a positive effect. Some factors that may affect this are time required for the lesson to be disseminated and time for changes in the training plan to take effect. You can discover observed behavior in plan and program reviews, event AARs, project reports, retrospectives, and summaries.
Lessons learned (and applied) are re-examined during active collection by the LL activity. Collection teams can look for older issues and determine if the problem still exists or if the problem was solved by information disseminated in the LL program.
Organization or unit performance
Verify LL and lessons applied by charting organization or unit performance against established measures of performance (MOPs) for mission-essential tasks that must be performed by the organization or unit. MOPs are simply metrics of tasks the organization or unit must be able to perform with its organic resources to accomplish its mission. Examples of this are enough trained drivers; personnel properly trained and certified on equipment; and the ability to perform operational, logistical, and administrative tasks in keeping with organization policy, regulations, and, if applicable, civil laws.
Lesson learned and lessons applied effectiveness can also be judged by charting organization or unit performance against measures of effectiveness (MOEs). MOEs are more goal and objective centric. The chief criteria for MOE success are the questions, "Is the organization effective in the accomplishment of its mission, and is it supporting the overall goals, objectives, and missions of the parent organization?" If an organization has increased mission effectiveness over previous performances, you can attribute it to lessons learned and applied from the LL activity. However, there may be other factors that account for an increase in mission effectiveness; therefore, additional collection may be required to verify the effectiveness of the LL program.
Establishing written procedures
Once you have decided the size and scope of your LL program, it is a good idea to formalize your program by developing a set of written guidelines, responsibilities, policies, and procedures. The military would call this a standing operating procedure. Whatever term is used, it is necessary to standardize your procedures in writing for several reasons. First, it is one way to get decision makers and leadership to agree to the specifics of the program. Second, it is a document that everyone can read and become familiar with all procedures. Third, it allows participants to understand their specific responsibilities. Fourth, it can be used to support funding decisions, since you will have established procedures approved by your executive leadership. Finally, it gives you a plan that can be adjusted over time to accommodate changing priorities or direction in your organization or unit. The goal of written procedures is to enhance unity of effort.