CALL title banner
Handbook 11-33
June 2011

Chapter 2. Functions of a Lessons Learned Capability


Lessons learned (LL) programs will vary according to the needs of the organization; however, most will have at least six functions as shown in Figure 2-1. Each of these functions is explained in greater detail in this chapter.



Collect

Analyze

Share

Archive

Resolve

Assess


Figure 2-1. LL functions



Collect

The first function of any LL capability or program is the collection of information relevant to an issue that someone has determined requires analysis. In an LL program, the issues are usually topics the organization realizes are causing concern or problems, and the organization wants to determine a better way to do business or enhance performance. Historically, most organizations are passive when it comes to reporting problems and potential solutions shared with other like organizations so those other organizations do not encounter the same difficulties. Normally, if someone does not make the effort to "pull" these issues from the organization, do not expect them to "push" them to you. Many reasons exist for this, such as operational pace, shortages of manpower, time constraints, a lack of understanding the importance of sharing information, or no process in place that facilitates the sharing. Ultimately, the goal of any collection effort is to gather enough information to have informed analysis that resolves the issue so other organizations can benefit from the experiences of those who have gone before them.

We do not collect lessons to evaluate an organization; rather, we collect lessons to help organizations improve. If perception of the LL program is as an evaluator, an inspector, or an agency of internal review, no one will be willing to share problems and corrective actions. A policy of nonattribution may be appropriate. Make every effort to avoid personally naming individuals; instead, use duty positions or work positions. Try to avoid naming specific units or offices if possible. For example, if you were collecting on the 1st Battalion, 502nd Aviation Regiment, instead of saying 1-502 Aviation, simply say an aviation battalion. In other words, be generic when it comes to sources but specific when it comes to issues. To get the desired access and have credibility with your organizations, the LL program must be seen as trustworthy, unbiased, and able to guard sources when asked to do so. That said, information gathered from a collection should not be watered down or altered. Collect the facts and only the facts, and try to avoid opinions.

Finally, there is a tendency in any LL program to focus on what is not working well (the negative). A good LL program should also collect information on what is working well (the positive). "Best practices" are what we call positive observations. Certainly, the tendency is to focus on the problem areas, but you may find organizations that have mastered a certain task and now can share those best practices with others. As a rule of thumb, expect that 80 percent of what you collect may be negative (in an attempt to improve); however, 20 percent should be positive best practices.

Collection opportunities

Opportunities to collect information on issues conform to the mission of the organization the LL program is supporting. For example, a military organization may have the responsibility to protect our nation from foreign adversaries but also provide support to civil operations in times of national emergency. When you analyze that mission, there are many collection opportunities that exist that have the ability to provide vital observations and lessons on how to best conduct those operations in the future. Although the participants should be the first line of observation reporting, often it becomes necessary for the LL organization to be proactive in the collection process. The following is a potential list of collection opportunities:

  • Training events and exercises that prepare the unit for each of its specified missions.
  • Experiments and testing that support the unit's mission.
  • Planning sessions and conferences that support the unit's mission.
  • Observing the actual event or mission.

As you can see from this list, there are several opportunities to gather information. To get the maximum benefit from these opportunities and match them to the available resources requires a long-range collection schedule be developed that overlaps the various opportunities and phases them in over time so the workload is manageable. Typically, this schedule runs for one year and makes an effort to lock in dates and locations for each collection effort. In the example above, the occurrence of a natural disaster is difficult if not impossible to plan for; therefore, the LL program must have the capability to initiate impromptu collections on very short notice. Independent LL organizations may need a collection schedule; however, organizations with smaller, internal LL programs may not.

Determining critical issues

One of the greatest challenges of the collection phase is determining what exactly are the issues that require a collection effort be analyzed. There are several ways to accomplish this. Often issues are "command directed." A higher headquarters, agency, or organizational leader directs you to collect on a certain issue of interest to them. This makes life simple and usually allows you to avoid a deliberate process whereby the LL program must decide what issues are most important. Sometimes a "shotgun" approach is effective. This approach understands that in every complex operation there are always issues that require work, and if the LL program is working, the issues will certainly be discovered. The disadvantage with this approach is that you do not focus and you have a tendency to look at everything, which dilutes your efforts.

Another method might be the feedback gained from multiple units or organizations that have expertise performing a certain function that is not working properly. When numerous experts in the field are all reporting the same issue, it is probably critical. For example, after the initial invasion of Iraq in 2003, the U.S. Army experienced problems defeating the rocket-propelled grenade that was destroying the new Stryker combat vehicle. Because of this immediate feedback from commanders in the field, the Army devised a defensive shield that could be constructed around the vehicle to detonate the missiles prematurely, rendering them ineffective.

Finally, a more analytical and deliberate approach would require some degree of analysis at home station to determine the issues that are most important to your leadership. A deliberate approach focuses your time and resources but requires a decision process to be developed to determine what is critical. One way to do this is to study past operations, reports, articles, and briefings and begin to compile a list of issues that seem to reoccur and cause difficulty. You can weight each issue with its degree of importance, assess risk, rank order the issues, prioritize resources, and make a recommendation to senior leadership on which issues are most important for the purpose of collection. Each collection effort will more than likely uncover new issues that can be added to the existing list. Once again, you will need to go through a process to determine which issues are most critical for future collection efforts.

The collection plan

Once you have determined the critical issues, you will need to develop a collection plan to guide the collection. A collection plan can be as simple as a list of questions you desire to ask the organization and its subject matter experts (SMEs) or as detailed as the following:

  • Specific guidance for the observers.
  • A delineation of collection responsibilities for each observer.
  • A list of questions for each issue.
  • A determination of the collection methodology for asking each question, which includes who will be asked each question.
  • Identification of documents and resources to be used in conjunction with the collection effort.
  • A schedule of interviews.
  • Travel, support, and information management.

However, the common threads in all collection plans are the questions. You never answer a good question with a yes or no. The question should be worded in such a way that the person answering the question must give a full account of the process the organization is using. Follow these examples:


WRONG: Is your unit performing route clearance operations?

Are we measuring performance improvements in the targeted areas?

RIGHT: How is your unit performing route clearance operations?

How might we measure performance improvements in the target areas?

As stated previously, you must avoid the tendency to collect on too many issues. More is not necessarily better. As a general rule but based on the size of your collection team, more than 10 issues for any one collection effort gets cumbersome. If a collection team can deploy with six to 12 lead questions already prepared for each issue, that normally gives you the degree of fidelity you need to get good results. Additionally, the collection plan is only a plan. Once you begin to ask questions to the interviewees, there is nothing wrong with developing new questions "on-the-fly" that better address the issues based on the information you are gaining from the unit's SMEs. Do not get locked into the plan. It is only a start-point to get you moving in the right direction.

Finally, there is another collection plan technique you may or may not elect to use. It involves the use of a hypothesis statement for each issue. The hypothesis statement is a statement that defines what you want to confirm or deny about the issue. For example:


Hypothesis: Brigade combat teams continue to have difficulty clearing routes due to the enemy's use of improvised explosive devices (IEDs).

In this example, there is probably some anecdotal information that indicates units are experiencing problems with this enemy tactic. Focusing on the collection plan will confirm or deny this issue. The use of a hypothesis statement is sometimes more beneficial in focusing your collection efforts when you are trying to confirm that a corrective has been applied and a change of behavior is anticipated for the better. For example:


Hypothesis: The implementation of counter IED-defeat mechanisms has greatly reduced the threat of IEDs along routes traveled by brigade combat teams.

Figure 2-2 gives a sample collection plan format for briefing purposes on one issue. Again, the plan can be as detailed or as simple as the observer desires. The sample shows the issue and hypothesis, lists lead or key questions to ask, and describes the interviewees or individuals who will be asked the questions. Appendix A gives an example of a very detailed collection plan, and Appendix B gives an example of interview techniques.

Direct and indirect collections

Now that you have a collection plan, you must begin to decide (if you have not already done so) how you intend to conduct the collection. An LL program will normally use two different types of collections, both accomplishing the same tasks but each using different resources. The direct (sometimes called formal) collection effort always involves detailed planning and is generally a top-down driven process. The indirect (sometimes called informal) collection effort uses much less planning and generally fewer resources, but is typically a bottom-up process.

An example of a direct collection would be a team of SMEs assembled and trained on the collection process and how to build a collection plan, focused on specific issues generated by a higher authority, and deploying overseas to complete a 30-day collection on a specific organization. An example of an indirect collection could be as simple as a Soldier in a military unit submitting an observation he wrote to his headquarters, which forwards it up the chain of command, eventually getting to the LL organization. This is sometimes called an unsolicited observation. It may or may not be tied to an existing issue, but it is information that someone deems critical enough to take the time and effort to submit in writing from a lower echelon. Although it is always challenging to get organizations to submit unsolicited observations, they are typically some of the best lessons an LL organization receives.



  • Issue: Field artillery (FA) core competencies
  • Hypothesis: FA personnel participating in Operation Enduring Freedom (OEF) and conducting nonstandard missions are degraded in their FA/fire support core competencies.
  • Lead Questions:
    • What nonstandard missions did you conduct?
    • How much of your time was spent on other than FA mission sets?
    • How much time are you spending on conducting FA missions?
    • How has this experience impacted your primary skills of providing fire support?
    • What can Fort Sill do to help units doing nonstandard missions?
    • What sort of predeployment training would have better prepared you for nonstandard missions?
  • Interviewees: FA enlisted Soldiers, sergeants, and lieutenants in a maneuver battalion; FA enlisted Soldiers, sergeants, lieutenants in an FA battalion on a brigade combat team; FA battery commanders; FA battalion S-3 officers; and FA battalion commanders.

Figure 2-2. Example collection plan briefing



Another informal collection technique may be simple research. If a qualified analyst or researcher conducts extensive research on an issue and finds the right information and lessons, it may prevent the requirement for a direct collection effort, saving time and resources. In either case, the researcher or observer in each instance should have some degree of expertise in the area he or she is exploring, hopefully an SME who can provide valuable insights into the operation.

The only reason for understanding the distinction between direct and indirect collections is to realize there are many ways to get LL. Some ways are more challenging than others. It may sound difficult to put together a direct collection team, but it is sometimes harder to get feedback from the field. Anyone involved with LL programs for any amount of time understands this. If timely feedback was so prevalent, there would be little need to deploy direct teams.

Another technique for collection could at times be direct or indirect. It concerns the use of embedded teams or liaison officers. The U.S. Army has effectively used this concept during the Iraq and Afghanistan wars. They brought Reserve officers on active duty, formally trained them on collection techniques, helped them develop a collection plan, and then deployed them into theater for periods of six months to one year. Each officer was embedded with a unit or a headquarters. They provided vital and timely information on unit operations; tactics, techniques, and procedures; and lessons learned. This effective "push" system was probably a direct collection due to the planning and resources involved. Other techniques might involve the use of survey instruments, or the positioning of liaison officers with various organizations that have their own LL programs, to maximizing the sharing of lessons across agency boundaries.

The military uses a very instructive process as outlined in Appendix C called the after action review/report (AAR). You would think that this is an indirect process but one that requires a great deal of effort and time to execute. This process is one that produces some of the best and most timely lessons.

  • The after action review is a verbal discussion held at the completion of an operation or event with key participants to determine what happened, what worked, what did not work, and how to improve for the next event.
  • The after action report is a written document that highlights unit accomplishments and LL. It works well as a historical reference tool and is given to other organizations getting ready to participate in the same event.

Although the U.S. Army developed these concepts, with some modification they could be adapted for interagency, intergovernmental, or civilian uses.

Since the direct collection is probably the most complex to execute, it is beneficial to understand the process to execute this task. Figure 2-3 depicts an example of what a direct collection team may look like. This example shows four observers/SMEs, but the team could be as many as 12 members if the issues support that size. More than 12 are cumbersome, especially if you are traveling overseas.



Figure 2-3. Generic direct collection team composition



The team chief has the responsibility to lead the team, validate the collection plan, review observations drafted by observers, meet regularly with the team to review progress, assist in writing the observation report, and conduct any briefings required to introduce the team or provide an out-briefing when the team departs the organization it is observing. The team chief should be of sufficient position or rank to "open doors" if the requirement exists. In the military, the team chief is usually a colonel.

The operations officer is the team chief's deputy and his "right-hand man." He is usually a full-time member of the LL organization with prior experience supporting collection teams. He performs administrative tasks such as equipping the team, making travel arrangements, and scheduling interviews, and helps the team chief review observations for correctness.

Observers/SMEs make up the remainder of the collection team. Observers have the responsibility to be objective, do unbiased reporting, have no personal agendas, and be good writers, since they are responsible for drafting all of their observations for the observation report in the formats described below. Besides the use of a survey tool, the primary way to get information as an observer is through an oral interview. The observer should be an SME as well as the person he is interviewing. Information on how to conduct an oral interview is in Appendix B.

Training for any collection team is important. Normally, three or four days of training before the team departs is sufficient. Subjects taught in this "predeployment" workshop are observer duties; security training; collection plan development; and any training required on issued equipment such as laptop computers, cameras, satellite phones, and voice recorders. However, the main purpose of the predeployment workshop is to finalize the collection plan to the team chief's liking. It is also important to conduct a "post-deployment" workshop. The primary purpose of this workshop is to draft the observation report as described below. A secondary purpose is to return issued equipment, finalize travel arrangements, complete a security debriefing to ensure there is no compromise of classified information, and give a commander's or director's briefing.

Observation reports

The result of a direct collection effort should be a well-written observation report. Although modification may be necessary to correspond to any situation or desired format, the following sections are included:

  • Introduction: The introduction may discuss the mission of the collection team, dates of the collection, and who was involved, and typically thanks the organizations observed for their support. It gives any other special instructions and may highlight the classification of the report.
  • Executive Summary (EXSUM): The team chief leading the collection, with input from the individual observers, usually writes the EXSUM. It summarizes the results of the collection by issue. A good technique in the EXSUM is to list the overall top three or four issues that require immediate attention. Additionally, the EXSUM should cover what is working well. If someone reads only the EXSUM, they should get a good sense of what the collection was about and the results.
  • Chapters: There should be one chapter for each collection issue. Each observer responsible for an issue should write his or her own chapter. The chapter should start with a brief paragraph summarizing the major findings in this area. It should then list each observation and the associated lessons, recommendations, and any best practices in the prescribed format.
  • Appendices: Miscellaneous information such as maps, charts, checklists, photos, operation orders, and briefing slides can be included here.

You can use several different formats to write an observation. One of the easiest to use is the issue-discussion-recommendation or observation-discussion-recommendation format. Another format the U.S. Army uses is the following:

  • Title: The topic of the observation.
  • Description: One sentence explaining the observation.
  • Discussion: The major analysis of the problem supported by facts and examples.
  • Lessons: A list of any lessons observed to share with other units supported by the discussion.
  • Doctrine, organization, training, materiel, leadership and education, personnel, facilities (DOTMLPF) Recommendations: Recommendations for corrective actions under each category that applies supported by the discussion.

The joint community uses the following observation format:

  • Topic/Issue: The title of the observation.
  • Observation: One sentence explaining the observation.
  • Discussion: The major analysis of the problem supported by facts and examples.
  • Recommendations: A list of recommended corrective actions supported by the discussion.
  • Implications/Lessons: A statement of what happens if the recommendations are not adopted.

NATO uses the following observation format:

  • Title: Brief but specific description of the topic.
  • Observation: Brief factual description of what happened.
  • Discussion: What is relevant to determining the root cause? (May include history of the event and the what, where, when, why, who, and how.)
  • Conclusion: Summary of the underlying issue arising logically from the observations and discussion.
  • Recommendation: Specific actions that should be taken and by whom to deal with the underlying issue presented in the conclusion.

NATO also has a format for a best practice:

  • Title: Topic of the best practice.
  • Context: Sets the context for the best practice and the problem area or situation that it resolves or improves.
  • Process: Covers the main body of the best practice; an explanation of how to execute the best practice, and what processes and products it requires.
  • Benefit: Justifies why it should be classified as a best practice. (Provide a clear reason why it saves lives, time, money, resources, etc.)

The key to any format is that you thoroughly discuss the observation using facts and examples while trying to avoid opinions, highlighting any lessons and making recommendations to fix the problem if you have the expertise to do so. When dealing with LL, do not mistakenly write a truism instead. A truism is something we know to be true at all times. At one point, it may have been an LL (a long time ago), but over time, it becomes enduring. For example, IEDs are a major killer on the battlefield. Years ago, at the start of the war on terrorism, this was a lesson learned; today, it is a truism. The LL, in this example, would be ways to defeat IEDs.

A good length for a written observation is one to two pages. Another thing to keep in mind is that the "shelf life" for an observation is about six months. This means that anyone who attempts to use the observation to justify an effort after about six months needs to confirm with the source that the observation is still valid. For example, maybe the source organization determined the observation will not be corrected for whatever reason, or maybe the observation was corrected and it is no longer an issue. Using an old observation or observation report that is no longer valid to substantiate or justify a project without making sure it is still an accurate assessment of the issue will show that you have not done thorough research.

For this reason, some LL organizations prefer to keep raw observations as internal reports, disseminated only to those who have a need for the information. One technique to use is putting a statement in the front of the report that "warns" the user after a certain period of time to make sure they verify the status of each observation before citing them in another project. If you have the time and energy, a very good procedure is to update the status of each observation once any new information develops concerning its resolution. This requires a conscientious staff and considerable time and effort to monitor and track the actions associated with each observation, most of which may not directly involve the LL organization. The issues- resolution process, which will be discussed later, can provide information to support this effort.

Make every effort to keep the observation report classified at the lowest level possible. That said, it is the responsibility of the observer to ask the interviewee the sensitivity of the material he is receiving. If there is any doubt, always have the unit intelligence officer or facility security specialist confirm the classification of the information in question. Be especially careful with photographs taken or briefing slides with embedded pictures. Organizations that deal with classified information on a daily basis are sometimes sloppy in their handling and marking of documents.

Finally, out of courtesy, it is appropriate to let the organizations interviewed review a final draft of the observation report. In some instances, they may make recommendations that clarify what was given in the interviews. Typically, they should have at least two to three weeks to review the observation, a time you need to factor into your planning sequence.

What you must avoid is the tendency for the reviewing organization to "correct the report." Any LL program has the responsibility to present only the facts. One way to compromise on points of tension is to include the unit's opinion as an appendix. This should keep both sides happy.

Figure 2-4 highlights the LL process and where the collect function fits.



Figure 2-4. The LL process with the collect function



Analyze

Analysis is a process used to thoroughly understand areas of activity identified to have potential for improvement. You can accomplish this action at the end of the event or after the active collection has been completed; however, a certain amount of analysis may be done throughout the LL process. Transforming the raw data into actionable recommendations requires a systematic process to examine the information that has already been collected and understand why or what contributed to the need for improvement. The level of analysis may be determined by the expertise or resources available or by time limitations in developing a final product. In some cases, it may be more important to conduct a surface analysis and expedite the results back to the user so they can start to take corrective action, where as with other observations, the complexity of the issue or resolution may necessitate a more detailed analysis and explanation. In either case, raw observations may change in context, content, conclusion, and applicability during analysis.

Validation of the observation

During this part of the analysis, you start to organize the data you have collected, ensure you have explored every possible resource, and agree on the direction and method the analysis should follow.

Frequency of occurrences. Review previous observation reports, AARs, LL databases, and other reports to determine the frequency and conditions in which the observations have occurred. Is there a solution on record? An effort should be made to determine if the issue has been previously identified and a solution initiated but not yet implemented.

Understand the objective. It is important to understand where to focus your analysis. While conducting the analysis, it is easy to identify additional issues that may cause you to stray away from the intended objective or the customer's needs. Consider capturing these issues and setting them aside for later collection efforts.

Review findings with the host organization or other stakeholders. If not already accomplished during the collection phase, review the observations with the unit or organization where it was collected to determine if there were any unusual or contributing conditions for the issue. Do not provide recommendations at this point, since they may be based on incomplete analysis.

Analysis of observed data

This is the step where the analysts start to analyze, brainstorm, and dissect the information collected. Use different perspectives when looking at the data to fully understand the issues, examine each piece to see if other issues exist, and start to develop the full story. It is critical to discover not only what happened but why it happened.

  • Conduct additional research on the observation(s). This may require making follow-up interviews or phone calls, reviewing AARs or other collection reports, and searching existing LL databases for related information. Talking to other SMEs who are knowledgeable of the issue may also provide some missing information or confirm the data you have already collected.
  • Seek expert consultation on the issue to help you understand the data you have collected, identify contributing factors, and help make informed recommendations. In most cases, you will not be an SME on the particular issue, so it is important to find someone who can answer questions, review the observation and recommendations, or participate in the analysis of the collected data. If the issue is a piece of equipment or system, it may be necessary to contact the manufacturer or designer. If it is a training issue, you may want to contact the training center or proponent for the issue to see what is being taught and what is missing.
  • Organize group discussions or "murder boards" to examine the observations in more detail, identify capability gaps, understand the root cause of the finding, and determine if there are related issues. Organize these discussions with someone designated to keep the discussion on track and the analysis of data moving in the right direction. Designate someone to record the results of the discussion. Displaying the data on a screen can be useful for the group to view and agree on the outcome, but it can also cause the discussion to get bogged down in "wordsmithing" and delays in the final product.
  • There may be a requirement to conduct some basic statistical analysis of answers you received in questionnaires or interviews. It is important to interpret and sum up the results for the purpose of the analysis and final report. Model the patterns in the data in a way that accounts for randomness and uncertainty in the observation. It may be useful to display this information in the form of charts or graphs in the final product to emphasize the frequency of an observation, gaps in a capability, or need for a particular solution.

Corrective actions/recommendations

During this phase you start to organize the results of the analysis and determine appropriate recommendations. If you do the collection and analysis correctly, the recommendations should be intuitive. The recommendations should say what needs to be done and not just what effect needs to be achieved.

  • Prepare the results of the analysis in a format that is accurate and easy to understand. It may be best to first capture the data on briefing cards or in briefing slides to help organize the main points in a logical order. Add to the observation report once the data is developed and organized, as described in the Collect section above.
  • Provide realistic, actionable recommendations. Making recommendations that are not feasible may only slow down the process of making any corrections. Try to designate the agency or organization that should lead the corrective action. It may also be helpful to include an implication statement or what the impact would be if the situation is not corrected.
  • Support any conclusions with interview transcripts, statistical data, or other documents that will provide credibility to the observation and help facilitate the solution process.
  • The analytical process transforms initial observations into best practices and lessons by grouping common observations to organizational functions. As previously mentioned, the military uses the DOTMLPF construct to group solutions; however, other categories may be more suitable for other organizations.

The end state of the analysis process should be a well-defined list of recommendations or potential corrective actions with sufficient detail to be shared, achieved, and entered into the issues-resolution process. If done correctly, the analysis will help determine the action plan and resources required to implement the corrective actions.

Figure 2-5 highlights the LL process and where the analyze function fits.



Figure 2-5. The LL process with the analyze function



Share

LL programs must have the ability to share and disseminate information to be effective. Besides the ability to share information, the LL program must be able to determine what information is important or urgent and how rapidly it must be passed to other organizations that could benefit from the knowledge. The LL program must have a process and medium to do this. You can share through several means, such as printed "hard copies," electronic forms like e-mail and messaging, collaborative forums, and websites. The process should support the capabilities of the medium. Additionally, you should have the ability to handle both classified and unclassified material.

Resourcing of the LL program will usually determine what dissemination means are used; however, it helps if the resourcing conforms to the information required by the target audience to successfully execute their missions. For example, printing professional, hard copy documents can be expensive and may exceed the budget limits of the LL organization, but some deployed units who are the target audience may not have access to computers, printers, or copiers in a field environment and need hard copy materials to do their mission effectively. In the future, social networking sites similar to Facebook, Twitter, and LinkedIn may provide additional capability to share lessons if they can be adapted to the requirements of an existing LL program. Finally, it is imperative that all LL organizations within a common community be linked to facilitate the flow and rapid exchange of information.



Prioritization of Information

The key to information or LL dissemination is a rapid sharing process. This requires an ability to rapidly analyze information from collections, determine relevancy and timeliness, and gain permission from leadership to share. The challenge in this process is the quicker you need to get the information out, the more risk you assume in conducting a thorough analysis to make sure you are drawing the correct lessons.

An example of one way to construct a rapid sharing process is depicted in Figure 2-6. In this example, the LL program has created timelines for sharing that are tied to the urgency of the information and a medium to disseminate that information. The terms immediate, urgent, and routine would need to be specifically defined to meet the goals of the supported organizations and their mission. For example, in the military, an "immediate" requirement to share a particular lesson, if not shared, may result in the injury or death of a Soldier. However, once rapidly shared, information should continue through the analysis process and eventually be formally vetted, archived, and become a part of the issues-resolution process, if it rises to that level of importance.

You can publish lessons of less priority as articles, reports, bulletins, and so forth. The U.S. Army has a medium where any Soldier in an operation can write an article that can be posted on the LL program website for anyone to read. It is called "News From the Front." This is an excellent way to post timely, thought-provoking pieces that discuss best practices; lessons learned; and tactics, techniques, and procedures. An observation report, discussed in the Collect section, requires a longer lead time to produce because it is usually part of a direct collection effort involving a deliberate planning process. Finally, the production of a newsletter, handbook, or other periodical that may or may not go to a print plant for production usually requires the greatest resources and the longest lead times to produce. They may require anywhere from three to six months to develop and print; however, they additionally serve as excellent historical documents that may be very beneficial in supporting future operations.



Figure 2-6. Example rapid sharing process



Publications and automation

As previously mentioned, the production of publications and the ability to disseminate information may be a consideration for any LL program. If you cannot share the information you collect, it is virtually useless. Some of the major functions of a publications capability may be the following:

  • Manage the publications process from first draft to final product to ensure consistency, clarity, and correctness.
  • Ensure funding is available.
  • Retain necessary editorial and graphic artist support.
  • Accomplish all staffing actions before final publication.
  • Coordinate with the printing office and print subcontractors to ensure a quality product is disseminated in a timely fashion.
  • Work with proponent and supporting agencies to coordinate solutions and exchange ideas on production efforts.

Some examples of categories of potential publications based on subject matter, size, and frequency are:

  • Handbook: A "how-to" manual on a specific subject that generally takes a longer time to produce than most other publications.
  • Newsletter: Normally less intensive than a handbook to prepare but focused on a specific topic of interest that may include opinions from a wide range of SMEs used to stimulate discussion and thought.
  • Periodic Bulletin: Published weekly, monthly, quarterly, etc. Less intensive to produce than a newsletter and used to update the audience on information that is continually evolving or changing.
  • Article: Prepared by an author to highlight a specific topic of interest.
  • Special Study: A publication related to a specific operation, exercise, or subject, such as a country's history, environmental cautions, cultural do's and don'ts, and emerging doctrine or policy.
  • Observation Report: Published after the completion of a collection effort that summaries the observations, lessons, recommendations, and best practices from the collection. An AAR would also be another form of an observation report the LL organization could publish.

Before publishing, one challenge to sharing information is the degree to which the LL program wants to edit material. There will be times when the editorial process should be eliminated or modified to facilitate the expeditious release of information critical to the success of the organizations being supported. As the saying goes, "perfection is the enemy of good enough." The editorial process should not become so burdensome that it prevents a document from being published in a timely manner. On the other hand, if time is not an issue, a professional product is always a good option.

Automation obviously enhances the capability to share information. Although some organizations may prefer to retain a paper-based capability, that capability is usually expensive. Others may prefer a purely automated dissemination system using electronic distribution techniques such as e-mails, websites, and collaborative sites. Some organizations may prefer both capabilities, paper and electronic. However, making a portable document format (PDF) version of a document allows for easy document exchange and is low cost once the initial investment in automation is made.

You must give consideration to what extent the LL program wants to handle classified information. However, if your goal is widest dissemination, it is best to avoid over classification. Some LL programs have the ability to create databases for cataloging observations and lessons that can be shared with other organizations. Not only can these databases store observations, but also they can archive the source material and reports that support each observation. For example, the joint military community uses a system called the Joint Lessons Learned Information System (JLLIS), and the homeland security community uses a system called Lessons Learned Information Sharing. In any event, the challenge of publishing information and sharing it rapidly reduces by the sophistication and level of automation existing in most organizations and agencies today.

Figure 2-7 highlights the LL process and where the share function fits.



Figure 2-7. The LL process with the analyze function



Why don't people share?

  • It's not convenient.
  • They do not know what they know.
  • They do not know the value of what they know.
  • They believe knowledge hoarding is job security.
  • They do not get credit for it.
  • They do not have the time.
  • They do not know how.
  • They do not know who is interested.
  • Sharing of issues sometimes leads to additional work to fix.

If your LL program can begin to break down the barriers to sharing that are listed above, then you will be on your way to increasing the productivity and performance of the organization. Today, there are organizations and companies that have done just that.



Archive

The archive function is a broad term that encompasses several required capabilities that most LL programs desire. The ability to archive information and manage records, both print and electronic, allows access to them at a future date. To accomplish this task, digital repositories (which may be called digital libraries or archives) must be developed to store information, facilitate the historical preservation of information, and allow users to conduct research. Make it easily retrievable and available to any requestor by providing a logical system for storing information. Powerful search engines are required that permit rapid, user-friendly searches. Finally, repositories must have the capability to store and guard both classified and unclassified data.

Databases and websites

Databases and websites are necessary to store and access information. Some examples of what an LL program may desire are:

  • Stand-alone applications to build collection plans.
  • Stand-alone applications to save and catalog individual observations resulting from collections.
  • Stand-alone applications to build AARs.
  • Digital repositories to archive current information and/or older information.
  • Programs that interface with other LL organizations.
  • Organizational websites.
  • Collaborative websites to share information.

The LL program can take advantage of this technology to increase its efficiency and ability to access information in a rapid manner. One of the greatest challenges an LL organization will eventually face is the sheer volume of information that will accumulate in its archives. You will need to be constantly planning ahead and determining the best new software to use to upgrade your capabilities and be open to new technologies.

Research

The primary reason to archive information is to have the ability to conduct future research. Research today no longer takes place on library shelves; it takes place electronically using the Internet. Searchable electronic archives will generally be of three types:

  • Classified/Sensitive repositories.
  • Unclassified repositories with access limited to specific groups, such as members of select organizations.
  • Public repositories available to all Internet users.

Searchable databases provide the researcher easy access to the information required. This capability necessitates the use of highly refined search engines. The use of standardized filing and naming conventions, protocols, and accurately tagging documents with the right metadata facilitates the search. The metadata describes what is in the document that leads you to the material you are trying to find. To ensure documents are properly processed, you may want to establish a quality assurance check. Additionally, the LL organization may require the capability to store printed material, therefore requiring a "vault" or physical repository for storage. The vault is also a good place to store back-up compact discs of all digital material archived electronically.

It is also beneficial to determine criteria for inclusion of information in the repository. Some parameters are as follows:

  • The item is from an official approved source.
  • The item will add value for future research.
  • The document's handling restrictions are within the organization's permissions.
  • The security classification is within the organization's authority to store.
  • The document is not copyrighted, unless you have on file a written permission attached to the document.

The archives of the LL organization may be for internal use only. However, if outside agencies are permitted to gain access to the archives, they usually have read-only and download permissions.

Finally, the way information is cataloged or organized must be simple and easy to understand. One trap many organizations fall into is when these procedures are designed by information technology personnel and not by the users. To be simple and easily understood, the user must have input. The ability to quickly find what you are looking for is paramount to any research capability. Simplicity is the key; the fewer keystrokes it takes to find a document the better.

Historical

Archived information serves a historical purpose, though the archiving and adding to the repository should take place as soon as possible after the creation of the document or information. Years from now, future generations may desire to study the accomplishments of your organization. They may also find those historical examples applicable to current operations. Archiving material where it can be easily retrieved serves this purpose.

For example, when the U.S. Army first realized it would be involved in counterinsurgency operations in Iraq, it was able to draw on documents archived within its LL program from the Vietnam War. Since these documents all existed before the advent of the computer, over the years great efforts were made to scan this information into the Army archives. Having done this, great time and effort was saved when it became necessary to review the lessons of that war to see if they had applicability for the new conflict, which they did.

A more recent example concerned Hurricane Katrina and its impact on the Gulf Coast. The Center for Army Lessons Learned (CALL) searched the Army LL program archives before the hurricane hit shore, looking for information relating to the 1992 Hurricane Andrew disaster in south Florida. CALL then rapidly shared the lessons contained in those reports with Army units preparing to deploy and support Hurricane Katrina recovery efforts.

The lessons were instrumental in assisting units in their planning efforts. As a result, archiving information for future generations is a much-desired capability within any LL program. It will mean your hard work may get used more than once.

Requests for information (RFIs)

Every LL program should develop a system to answer questions or RFIs submitted to it from individuals outside the organization. Providing timely responses to questions is another way of sharing information. Some examples of this process were explained in the Share section of this chapter. The RFI system should be able to operate in both classified and unclassified communication networks. The system, to operate efficiently, should include an e-mail system, a workflow process, a document management application, a structured query language database, Microsoft Web services, and a Web interface. Once information is located to provide a response to the RFI, it can be sent by using links to the documents with the requested information. While anyone outside your organization should be able to submit an RFI, responses are provided based on the individual's rights as determined by their security classification or clearance. The key to any good RFI system is providing a timely response with some degree of analytical underpinnings, so the requestor gets the information in a manner requiring minimal analysis on his part, thereby making the information instantly usable.

Security

If your LL program handles classified information, it must have the capability to review, store, and archive classified publications. Reviewing documents to determine the correct classification instructions or to ensure that documents are properly marked is a time-consuming procedure, typically requiring a separate security office to perform these duties. It will normally require dedicated personnel specifically trained to execute these tasks. Due to the sheer volume of publications coming from various sources, there can be mistakes made by the originators whereby briefings, reports, documents, etc. are improperly marked, which could lead to a security violation or "spillage" of classified data over an unclassified network.

The LL program must have the capability to perform electronic "key word" searches on each document received to determine if classified information is embedded accidently in a supposedly unclassified document. Additionally, you should require the security team to brief and debrief all direct-collection efforts, so personnel are trained on the procedures to handle, store, and transport classified material. Scrub for classified information, and destroy or properly secure all notes and working papers once the collection is completed. Establish rules for the use of removable media such as thumb drives and compact discs early on to avoid loss or spillage of information not intended for public release. In addition, all laptop computers used by a collection team should be purged of all information once the team returns and is done with their use. Finally, consider other aspects of security, which include operational security to protect the organization's plans and procedures and physical security to protect the facility, equipment, and personnel.

Figure 2-8 highlights the LL process and where the archive function fits.



Figure 2-8. The LL process with the archive function



Resolve

The most challenging component of any LL program is establishing a process to legitimately resolve issues once analysis is completed and as early and quickly in the process as possible. Issues resolution beyond the organization is challenging for several reasons:

  • Issues resolution requires a "forcing function." This means it typically needs executive-level support and involvement. In the military, this translates into general officer or flag officer participation in the process. Ideally, you should involve leaders at all levels who are willing to dedicate time and effort to resolve issues at whatever level they occur.
  • Someone must determine which issues to tackle, and are they willing to invest time and resources (funding and people) to bring about a change in behavior. Do not resource LL programs to accomplish this task.
  • To resolve any issue typically requires the involvement of more than one agency or department.
  • Many issues require long lead times to resolve, especially if the issue requires a materiel solution.
  • You can make a strong case that the LL organization is not the best organization to lead this process, since it has a vested interest in the collection process and any follow-on efforts to determine if the corrective actions are working.

Whether or not the LL program has responsibility for the issues-resolution process, the organization's ability to change behavior by implementing a lesson is ineffective unless you observe that change and a determination made that the lesson is learned. In other words, the corrective actions have enhanced performance. To do this requires a deliberate process to commit resources, make decisions, implement those decisions, and observe the results. If the process is cumbersome and too difficult, expect the results to be less than optimal. If the process takes on too many issues, expect the system to become overwhelmed and collapse on the sheer weight of taskings to agencies that simply cannot handle the additional workload. It is better to resolve your top three issues than to attempt to solve the 70 or 80 observations from the last collection. Understand that the level that developed the issue generally has the highest interest in resolving it.

Not all issues require a formal process to resolve. In the military context, if unit commanders have the capability to correct an issue internally, they should do that. It is possible that during a collection effort the unit or organization becomes aware of an internal problem unknown to the leadership. The first objective is to handle the corrective action at the lowest level possible. The issues that rise to the next level of attention are those the unit or organization are unable to correct internally. Most LL programs will focus on these issues. They are issues that require assistance from sources outside the unit's chain of command. For example, a major modification to a vehicle system would be an issue that requires outside assistance and generally exceeds the ability of a typical military unit to correct.

The issues-resolution process requirements are summarized in Figure 2-9.



  • Senior leader involvement.
  • Determination to commit resources.
  • Assign responsibility.
  • Develop an action plan.
  • Implement corrective action.
  • Verify the lesson is learned.

Figure 2-9. Issues-resolution process requirements



Senior leader involvement

Senior leader involvement or executive-level participation in the issues- resolution process is the key to success. Without senior-level leadership involvement with the authority to task agencies to work issues and reallocate resources, the process fails. For example, successful issues-resolution steps in the U.S. Army required involvement at the three-star general officer level. This commander had the ability to task his subordinate two-star commanders with responsibility for their respective service branches and combat development processes to drive changes to doctrine, training, education, or materiel. If you cannot fix responsibility at the appropriate level, then the process becomes the end state. The process is not as important as the product. The product, in this case, is a corrective action that when implemented, changes behavior. To develop that product is not an easy task and always requires someone to take ownership at a level able to direct others to complete actions and hold them accountable for their progress.

Determination to commit resources

If you want an effective LL program and an issues-resolution process, you should at some point present the issues gathered and analyzed to a leader who can assist in the prioritization process and make decisions to commit resources to solve problems. There will be few if any corrective actions that will not require the expenditure of some resource, be it time, additional manpower, or money. You will quickly determine you cannot fix everything. Obviously, you can accomplish some things with fewer resources than others. For example, a change to written doctrine or policy typically requires less effort and expenditure of resources than a materiel solution for a piece of equipment or new operating system.

To determine where to place your efforts requires a prioritization process or an ability to rank the issues from most important to least important. This may involve a risk assessment. Normally, risk assessments are subjective, and qualitative analysis becomes the norm. In some instances, a military commander, based on his experiences, professional judgment, and "gut" instincts, can make a good assessment of risk. In other instances, the assessment requires the collective input and wisdom from a group of SMEs or other professionals. For example, the military sometimes uses a "council of colonels" to assist in making recommendations to more senior officers for final decision. Civilian organizations may convene a director-level working group or use a board of directors. Eventually, you reach a consensus or you are directed where to place your efforts based on a combination of risk levels and resource availability.

The key is to focus your efforts on the most important things that need to be fixed, where you can get the greatest "bang for the buck." For example, in a military organization, every effort is made to fix those issues that may have life-or-death consequences; they are considered high-level risks. However, the challenge is sometimes greater in prioritizing issues that do not have that clear distinction. The goal of this requirement is to determine what you have the ability to correct and then begin to develop a plan to do so.

Assign responsibility

The person in charge with the authority to commit resources needs to pick a lead agency to work the issue. In most instances, there will be supporting agencies that must assist the lead agency. It is rare in today's operational environment that only one agency can fix a problem by itself. Additionally, the LL program should not be the lead agency. The LL program supports the resolution effort by providing guidance and information relating to ongoing or subsequent collection efforts that may inform or assist the lead agency in its work. Typically, the LL program does not have the specific subject matter expertise or the resourcing to resolve major issues of a very important nature that require a formal action plan. Its focus should remain on continued collection efforts, analysis of information, and sharing the lessons with other organizations that may experience the same challenges as the agencies working to correct the issue. As stated previously, the first place an issues-resolution process will fail is by not having senior leadership involved in the process from the start. The second place it will fail is by not assigning responsibility to a single agency to be the lead for a corrective action.

Develop an action plan

Once a determination is made to solve an issue, an action plan should be developed by the lead agency with responsibility to work the issue. You should approve the plan, usually at the level that has authority to assign resources. The action plan summarizes the issue, weighs the risk, outlines a way ahead or timeline for resolution of the issue, and assigns responsibility. The plan can also specifically assign the resources required. It can be as complex or as simple as needed. The example at Figure 2-10 shows a simple "quad" sheet format for briefing purposes. Establish more elaborate and detailed timelines in separate enclosures.

Risk levels are more important in initial decision briefings to determine prioritization. If you use risk levels, you must define what each risk level means. Once those are established and accepted, you can replace them with other necessary information categories, such as required resources/ constraints, essential tasks, or proposed end state. You can use any combination of categories to convey the necessary information and satisfy the desires of the briefing audience, as required. The goal of the action plan is to track the progress and milestones of the lead agency toward resolution of the issue.



Figure 2-10. Example quad sheet action plan



Implement corrective action

Implementation of a corrective action is seldom the responsibility of the LL program. This, again, is one reason why senior leadership must be involved in the process. In some instances, training may be required before implementation. Implementing a corrective action could be a long and deliberate procedure well beyond the ability of any LL program to manage or control, or it could simply be a change to an existing policy or procedure. Like any solution to a problem, there will usually be costs to implement in terms of time, people, and/or money. For example, the U.S. Army categorizes change using seven categories: doctrine, organization, training, materiel, leadership and education, personnel, and facilities (DOTMLPF). However, changing doctrine is generally easier to do than changing an organization design, etc. The LL program exists to facilitate the change process by providing observations and lessons for analysis and consideration for incorporation into an issues-resolutions process whose goal is to enhance organizational performance.

Verify the lesson is learned

Verifying the corrective action or lesson is "learned" once again comes under the oversight of the LL program. It will require an additional collection effort to validate the results. Whether that feedback is gained by a direct or indirect collection effort, the process is not complete until someone determines there is a change in behavior as a result of the corrective actions applied to the problem. The LL program must consider these additional collections as part of its overall responsibilities and plan/budget for them. This requirement to verify the lesson is learned is one reason why the LL program should remain as a "player" and not the "coach" of the issues- resolution process. By doing this, the LL program can remain the "honest broker" within the process.

You can make a case that in the military community a lesson is never really learned because of many factors such as personnel turnover, level of training, and changing operational environment. From the perspective of a doctrinal or materiel solution, this may not be as true. However, what this does point out is the human dimension of military operations. When an organization is centered around people, as is the U.S. Army, the ability to change behavior is an ongoing process. This at times can frustrate an LL program, because it may begin to see the same mistake made over and over again, even after a corrective action was applied. This should not discourage the program or indicate it is ineffective. Any good LL program understands this and adjusts accordingly by continuing to provide the most accurate information possible, so decision makers can decide how they want to prioritize and focus resources to resolve new deficiencies while maintaining an ability to observe the success of implementing past corrective actions. The most important point to remember is that an observation does not become an LL until behavior has changed.

Figure 2-11 below highlights the entire LL process by function.



Figure 2-11. The LL process



Assess

There are several ways to determine if the LL program is effective. You can evaluate any LL program by the expenditure of resources against the desired results. It is difficult to determine through quantitative analysis. However, there are some ways to evaluate the effectiveness of an LL program that may validate and justify its need. Assessment of LL effectiveness can be broken into several components: organization behavior, organization or unit performance, and mission effectiveness.

Observed behavior

If the training, education, or testing plan/program is not exhibiting the same errors or mistakes identified and addressed by the LL program, it is reasonable to conclude that the LL program has had a positive effect. Some factors that may affect this are time required for the lesson to be disseminated and time for changes in the training plan to take effect. You can discover observed behavior in plan and program reviews, event AARs, project reports, retrospectives, and summaries.

Lessons learned (and applied) are re-examined during active collection by the LL activity. Collection teams can look for older issues and determine if the problem still exists or if the problem was solved by information disseminated in the LL program.

Organization or unit performance

Verify LL and lessons applied by charting organization or unit performance against established measures of performance (MOPs) for mission-essential tasks that must be performed by the organization or unit. MOPs are simply metrics of tasks the organization or unit must be able to perform with its organic resources to accomplish its mission. Examples of this are enough trained drivers; personnel properly trained and certified on equipment; and the ability to perform operational, logistical, and administrative tasks in keeping with organization policy, regulations, and, if applicable, civil laws.

Mission effectiveness

Lesson learned and lessons applied effectiveness can also be judged by charting organization or unit performance against measures of effectiveness (MOEs). MOEs are more goal and objective centric. The chief criteria for MOE success are the questions, "Is the organization effective in the accomplishment of its mission, and is it supporting the overall goals, objectives, and missions of the parent organization?" If an organization has increased mission effectiveness over previous performances, you can attribute it to lessons learned and applied from the LL activity. However, there may be other factors that account for an increase in mission effectiveness; therefore, additional collection may be required to verify the effectiveness of the LL program.

Establishing written procedures

Once you have decided the size and scope of your LL program, it is a good idea to formalize your program by developing a set of written guidelines, responsibilities, policies, and procedures. The military would call this a standing operating procedure. Whatever term is used, it is necessary to standardize your procedures in writing for several reasons. First, it is one way to get decision makers and leadership to agree to the specifics of the program. Second, it is a document that everyone can read and become familiar with all procedures. Third, it allows participants to understand their specific responsibilities. Fourth, it can be used to support funding decisions, since you will have established procedures approved by your executive leadership. Finally, it gives you a plan that can be adjusted over time to accommodate changing priorities or direction in your organization or unit. The goal of written procedures is to enhance unity of effort.



 

Last Reviewed: May 18, 2012

 
          |   Privacy and Security Notice   |     |   Accessibility Help   |   External Link Disclaimer   |   No Fear Act   |
 
|   U.S. Army   |   Tradoc   TRADOC   |   iSALUTE   | Ft. Leavenworth   |   Site Map   |   FOIA   |   USA.GOV   |   This is an official U.S. Army Site   |