| drew davidson |


Developer and Client: Two E-Learning Perspectives


Drew Davidson, Ph.D.



E-Learning projects most often occur with two main perspectives intimately involved; a developer and a client. Each project has at least these two perspectives as a part of the development process (and while this paper is focusing on e-learning, they seem to be a part of pretty much the development process in any field). They can be represented by two different companies, or two groups within one company, or any variation therein. For sake of simplicity, I will define these two perspectives in general terms and then refer to them globally as the developer and the client (even if they can be represented by groups and individuals with different titles and roles). Granted these perspectives can blur, but I've found that regardless of the project, it's fairly easy to identify each of them. To help illustrate this discussion, I will share two stories of projects on which I worked, one from each perspective.

But first, let me define what I'm discussing here. The developer is the perspective of the group or individual that is actually developing and implementing the project; from content-creation, technology building, pedagogy, etc.   Often it is a diverse team led by a project manager or producer, and sub-groups of programmers, engineers, artists (2D, 3D, video, audio, etc.), instructional designers, etc. Working with the developer is a client. The client is the perspective of the group or individual who has final approval on the project (and often is the group footing the bill for the development of the project and possibly the distribution of the final product as well). This is the person or company who determines exactly when the project is successfully completed and is now a product ready for public distribution. This perspective can be another company, a publisher, or another internal group within a company that has final authority on approving projects as they are being developed.

So, developers and clients are two symbiotic perspectives in the development process of an e-learning project. They need each other in order to get the project completed, but there is often tension as they share the same goal from different perspectives. Most of this tension can be explained with the infamous axiom that when you are developing a project you have three important relational variables - time, money and quality - and you can only have two of them. Or, in other words, across a development cycle you often have to juggle these variables. For instance, you of course want high quality, but you don't have much money, so you need to allot for more time.   Or you have to sacrifice some quality in order to get it done for the small budget and the short timeframe you have. So this can often put developers and clients seemingly set against each other within the realities of developing the project. But in the end, both perspectives have to come together in order for a project to be successfully completed.

That said, I would like to share stories from two projects on which I've worked, one as a developer and one as a client. First, I'll relate the development of the1999 Entertech project ( http://entertech.org ), a project on which I was the third, and final, lead producer on the project. Then I will describe the development of the 2001 new media Language Arts products ( http://www.hrw.com/language/index.htm ) for Holt, Rinehart and Winston (HRW). A range of e-learning projects on which I was the senior manager in charge of general oversight and the contracting out of the development work to companies and freelance developers. In both cases, I'm relating these stories from memory and within the context of comparing and contrasting these two perspectives found in the development of e-learning projects. I'm no longer directly associated with the companies mentioned and any factual inaccuracies are unintended and solely my fault, but I have tried to remain true to the actual process of developing these projects.

The EnterTech project was developed by HumanCode (HC), a for-client, turn-key development house that was based in Austin, TX, but has since dissolved as a company. At the time, HC focused on three overlapping markets; entertainment, education and e-commerce. The Entertech project was an e-learning welfare-to-work project aimed at teaching job skills and helping people without high school degrees or GEDs maintain and succeed at jobs in the high-tech warehouse industry. It was named a best practice in the 2000 Report of the e-Texas Commission: education, excellence, efficiency, effectiveness (http://www.e-texas.org/). It is a web-delivered computer-based training application that assesses users' abilities in real time. The dual server, three-tiered (browser, Apache/Perl middleware, Oracle database backend) browser application with custom Java applets and Shockwave was designed to enhance both asynchronous & synchronous experiences, and includes a complex content management system that allows for custom creation of content based on specific needs. The learning environment consists of a well-balanced mix between computer, class and group work. Throughout, a simulated work environment is used to give the learner's experience a narrative context.

When I took over as lead producer, the project had been in development for roughly a year and would then take another year to reach initial completion and implementation.   At this time, we were at the stage where we had an art bible in place, the backend system had been fully designed and most of the instructional design was set. We still were creating content to fit into our existing framework as well as working to develop a middleware interface with the database. Our client was actually two-tiered in a way. We worked directly with the IC2 Institute ( http://www.ic2.org/ ) a research group affiliated with the University of Texas at Austin. IC2 had written the grants to get this project funded by the state government, as well as formed a business coalition that consisted of around 40 businesses (Dell, IBM, Motorola, just to name a few). So, IC2 communicated with HC, but the business coalition was also approving the development of this project, with the coalition's representatives primarily passing their communications through IC2.

The development team at this time had a lead group consisting of an Art Lead, a Lead Instructional Designer, a Creative Writer, Two Programming Leads (backend and frontend), a Quality-Assurance Lead, a Voice-Over Recording Specialist and several Subject-Matter Experts (SMEs). Underneath the lead group were the various teams making up around 50 individuals across the development cycle.

Throughout our process we continually tested as we developed this project. We ran usability tests, focus tests, and technical tests as well as a beta test prior to implementation. I believe this amount of testing was a large factor in the success of the final project. Along with testing, I think considering the learning goals and outcomes from the onset of the project helped to better ensure that they were met through this course. The Instructional Designers and SMEs designed assessment rubrics and arranged the interactive media within a learning experience facilitated by an instructor. Students would go through a 3-week course and they were sent through a variety of activities, stories and games and were offered real-time feedback on how well they were doing. Supplementary texts were developed to give facilitators opportunities to highlight the ideas, and to give students more activities with which to practice. Also, characters were developed to represent different learning styles. So we had a character who had to write everything down while they were listening, another character had to be shown how to do it, another had to read about it, and another had to actually get in there and do it. This helped us as a development team to keep our instructions open and flexible to better help a variety of students.

Developer Challenges

A major technical challenge was to make the program work with limited, low speed Internet connectivity that was available in the computer labs across the state of Texas where this e-learning experience would be offered.   Some of these labs had broadband networks, but many had dial-up. So we had to develop a hybrid experience that would work on macs and pcs as well as the various browsers on each platform. It was hybrid in another aspect as well. We developed the experience so it could be delivered completely online, or it could also be delivered with the aid of a CD-ROM. In the former case, all assessment data and media assets would run across the Internet. In the latter case, only the assessment data would travel across the Internet; the media assets would be drawn from a CD-ROM.   This required a complex database to be set up to allow as much flexibility as possible in the course content while also tracking everything students did. Our programmers custom-developed middleware to pass all the data garnered in the frontend (which was html, flash, shockwave, quicktime and java) to the backend database, and vice-versa. Since this was all custom-built, there were numerous iterations. During our testing, we found that technical difficulties with Internet connectivity could occur and were beyond our control (so the e-learning experience would not work). To compensate for this possibility, we developed in-class group activities and exercises that would enable instructors to continue with the course until connectivity was restored.

A major content challenge was to create and maintain a narrative arc throughout the 3-week experience as well as have engaging games and activities that had pedagogical goals and measured outcomes. We accomplished this by having the creative writer and instructional designer work together on both. They were supported by SMEs when needed and worked to create an overall experience that had great learning moments throughout. With two tiers of clients, we sometimes had contradictory feedback. One time, this occurred during the development of a segment on safety that led to dialogue being written immediately before it was recorded in the sound booth. But, the overall plot of attending training at the start of your new job in a (virtual) warehouse made for a cohesive learning experience throughout.

But the most challenging aspect was the management and organization of the project. We had to juggle tons of media assets and create custom programming, as well as work with a large internal team along with two levels of clients. We succeeded at this by having a great lead group and enough assistant producers to help oversee the project to fruition. But the secret to our success was clear communication, good documentation and constant organization. We held regular meetings as a development team and had meetings with the client bi-weekly. We also had a project website that helped everyone involved touch base with the status of the project. And we created thorough design documents and technical requirements and art bible and stuck to them. Near the completion of the project, we once had to sit down with the client and develop a bullet-point list of all their feedback and then rank each point on a priority scale of 1-5 (with 1 being high priority and 5 being low). This was a great example of time playing a role. With only so much time and budget left, we had to decide what was really important in order to deliver the highest quality product.

Finally, we managed to craft a creative partnership with the client, and working together we were able to cut features here, add features there, and develop a project of which we were all proud. In the end, I think our clear communication was crucial to the process, but the content would not have been as effective if we hadn't focused on instruction and narrative together from the onset and then tested the experience throughout the development cycle. As the developer, we were able to successfully implement this project for our client.

Thus ends my story from the perspective of the developer, one that highlighted the development process and how we navigated a relationship with the client while creating the project for them.   Now, I'm going to relate a story from the perspective of a client to show the other side of the equation when it comes to developing e-learning projects.

HRW is a high school textbook publisher that develops texts in 5 areas: Language Arts, Science & Health, Social Studies, Mathematics and World Languages.   The company has a small new media group that manages the development of all the media products (CD-ROMs, DVDs, websites, games, Learning Management System, etc.) that are associated with the primary textbooks. My role as a Senior Manager in the New Media department was to engage and oversee companies and vendors to develop these media products for the Language Arts area of HRW. So anything that was Language Arts, but wasn't a book, was hired out to others to develop for HRW. Internally, we had editors and small technical and design teams to help provide content and continuity with all of our offerings across media and subject areas.

As you may know, the high school textbook industry runs on state adoptions. So, companies compete to get states to adopt their texts for a number of years running. And the bigger the state's population, the more lucrative the adoption is. At the time I was in the industry, the companies still primarily sold just the textbooks and the new media products came gratis with the books, but we were working to develop an integrated technology offering (including a Learning Management System (LMS)) that would be worth buying along with the texts.

During 2001, we had around 60 projects worth of new media work to be integrated with the Language Arts textbooks. To start, I had internal meetings with the Language Arts editors (there was actually a group of editors dedicated to content for the new media projects as well as the editors directly in charge of the textbooks). We met to ensure we were on the same page of what new media work we'd like to have done to complement the textbooks. Once I had a sense of what was needed, I began placing Request For Proposals (RFPs) to the development community. I did this through networks of colleagues from my development days and from my knowledge of the industry as a whole. Some projects fit into existing lines of past new media products (such as CD-ROMs or audio cassettes for the hearing-impaired) and I engaged the companies and contractors who had successfully completed this work in the past. I also grouped work together around the textbooks in an attempt to engage a large enough company to tackle everything associated with that specific textbook (like the website, games, CD-ROMs, etc). Concurrently, there were some products that spanned multiple textbooks (like a Spelling game) that I looked to get one company to develop for all the textbooks.

Client Challenges

We faced three specific challenges. One was aesthetic. With so many different projects and so many different developers and contractors, we continually had to work to have a unifying graphic design across all the Language Arts projects, both new media and textbooks, as well as have resonance with the look and feel of the other HRW subject areas. This was facilitated by having an internal Design lead who helped oversee all the Language Arts projects and was a part of the Design Team that worked on all of the projects across all the subject areas. Needless to say, even with all the Art Bibles and oversight, this was still a tricky issue throughout the design and development of 60-odd projects. A good example of this would be when we had two development houses; one working on a textbook website, another working on the CD-ROM. Even with the same art direction from HRW, they managed to design to drastically different looks, and the editors liked them both. To refine the best of both designs and create one consistent look for the related products took several meetings that included both development teams and HRW. The three groups worked together and synthesized the design of the products.

A second challenge was technical.   In 2001, the development of the HRW LMS was in its infancy so it was extremely difficult to have good specifications to share with developers in order to develop products that could share data with the LMS. To navigate this difficulty, we would make the decision at the start of a project whether or not LMS connectivity was going to be part of the product. This helped to smooth out the development of non-LMS projects (like a stand alone CD-ROM), but we continually struggled with the projects in which we attempted LMS connectivity (like a textbook website). In fact, one website developed was supposed to have this connectivity, but it was completed prior to the completion of the LMS, so we had to go without. During my tenure with HRW this was never fully resolved, but an internal process was developing to help solve it, plus once the LMS was fully implemented, the specifications would be in place prior to starting future projects.

The third challenge was that HRW had recently been acquired by Harcourt, who in turn had been acquired by Reed-Elsevier. So while working on our projects we were also navigating attempts at overall efficiency and resonance within these larger frameworks. The company was looking for ways to standardize as much as possible while still having creative work done. So, we often had decisions made from above that had immediate and direct impact on the design and development of our projects. The best way to manage this was to be as flexible as possible while pro-actively presenting solutions to any issues raised by the acquisition activity. Again, the issue of an LMS solution is a great example, Harcourt wanted to require this feature, but with already completed websites, it was decided to forego it for the time being.

Within these three overarching challenges, I had the continual project management challenges of working to keep 60 projects on time and on budget, so a lot of spreadsheets and tracking of deliverables and payments. While this was crucial to managing the projects, communication was key (isn't it always?), so lots of meetings and memos and email. I was in charge of accepting the work, but internally, I had to work with the editors, design lead and tech lead to reach agreement on what was acceptable and what needed to be changed or improved. And externally I worked with the developers to get realistic schedules and budgets as well as clear design documents and definitions of the work required. Together, I worked to bridge the internal and external perspectives to help make the development of these projects as smooth a process as possible. Throughout, the editors helped to adapt existing textbook assessment strategies to work with the new media products. The editors also helped to arrange the learning experience integrated the new media together with the textbooks within a classroom environment.

Throughout the year of development there were too many stories to relate them all, but here are a few. We had a company that bid low on a demo for a project, which won them the bid. They then turned in astronomically high bid for the project as a whole, which subsequently lost them the bid. We had a team of contractors who had never worked together struggle to start a project. It took us twice as long as we planned to just settle on a design. But with a little more hands-on art direction from the HRW , they developed and delivered an award-winning e-learning spelling game. We also had people move to different positions of responsibility, both internally and externally, causing some problems with continuity across the development of the projects. What held it all together was good communication, solid documentation and continual organization. In the end, all the projects were completed on time and on budget (an amazing feat actually) and the 2001 new media products and textbooks went out for the States.


Relating both of these stories helps to illustrate the similarities and differences of the developer and client perspectives and how they combine together during the development of e-learning projects. In terms of similarities, it's painfully obvious how good communication, documentation and organization is so important. Plus, overall organization of the ideas and decisions help to keep projects on time and on budget. And while I mentioned that the issues of time, money and quality often would lead to having the perspectives clash, I believe there is an inherent difference in focus that contributes to this as well. Developers are so well focused on the goal of their project, the creative design and development of the project is their main purview. The client manages at a less direct level with a focus on the overall goal of where the project fits into the bigger picture. These different foci often mean that both perspectives are actually looking at different things and clashes can arise.

That said, having worked as both a developer and as a client, I firmly believe that the design and development of e-learning projects depends on initially incorporating assessment and arrangement. In both the HC and HRW projects, assessment strategies were included to help best ensure that learning occurred through the e-learning experiences. Concurrently, the e-learning experiences were arranged within a pedagogical context that enabled instructors to help guide students through them.   If assessment and arrangement are not considered, the learning in the e-learning products will most likely not be as successful. So while the perspectives of developers and clients may differ, and issues around time, money and quality will always be present, assessment and arrangement should always be considered when designing and developing e-learning projects in order to best create successful e-learning products.



| drew davidson |