Frequently Asked Questions

You can submit your application between now and May 14^th, 2013 (11:59pm, US EST).  During that time, you may require clarification or answers to questions that others may have.  Please return to this page to seek guidance, or you may submit your questions directly to us at: [email protected]

We will continue to update answers to frequently asked questions here.

What is the purpose of the Classroom Trials?

The Classroom Trials are designed to test the capabilities of software systemsdesigned to support the instruction of writing in a classroom setting. We are limiting the products for consideration to those that include a feature of automated feedback to teachers, students and others supporting the instruction of writing in a classroom setting. The Working Group responsible for designing those trials is addressing an important challenge in the American public education system. Two emerging trends face the classroom teacher:  1) students are increasingly composing in digital environments; and, 2) demands on teachers to deliver meaningful assessment and formative instruction are climbing. Also, as classroom sizes grow and demand for writing increases, it is unreasonable to expect teachers to respond to student writing efficiently. As a result of such transformations, software systems have been developed to ensure that students are prepared to meet contemporary academic and workplace writing demands.  If software can help enhance meaningful review of student work, then those resources should be examined for their capability to support writing instruction.

Technologies exist today that are designed to provide teachers with tools to review student writing at each phase of the drafting process, to provide critical and timely feedback to students, and to provide some of the information teachers need to strengthen instruction. Those formative goals are key to improved instruction. However, many emergent technologies have not yet been examined in a fair, uniform and transparent way. So, the Classroom Trials are designed to demonstrate whether specific software systems are capable of:

1. Increasing the quality and quantity of writing by students;
2. Supporting teachers to meet identified performance goals for writing instruction;
3. Matching the needs of students and teachers within the constraints of typical school environments.

We do not intend to deliver a study to demonstrate long-term or longitudinal outcomes for academic achievement; rather, we have designed the study to examine the claims of specific technologies, to see if they can benefit broader objectives. We believe that if a teacher's job is more meaningful through technology, then the teacher will become a more effective collaborator in examining classroom applications of emerging technologies. Our long-term objective is more and better student writing, but we begin with the short-term goal of identifying where and how select systems can help teachers and students carry the workload towards improved writing and writing instruction.

Who is responsible for the Classroom Trials?

The Classroom Trials have been designed by a Working Group; those members are responsible for reviewing and scoring your application to participate, and they are each named on this website with detailed descriptions of their organization affiliation and background (see THE REVIEW PROCESS).  The Working Group has received generous support from the Carnegie Corporation of New York and the William and Flora Hewlett Foundation.  In addition, the Working Group has enjoyed guidance and support from twelve other grant making Foundations and education advocacy organizations (the Advisory Board).  The Working Group maintains close communications with such organizations as the U.S. Department of Education, the NEA, the AFT and state providers, to ensure that the operational requirements to deliver the Classroom Trials are aligned with federal, state, district and local needs.

Why should our organization participate?

Providers of software systems designed to support the instruction of writing are in need of a comprehensive study of the ability of the systems to support writing instruction. For many, success has been measured in increments, such as the number of students or teachers using an application. We seek to offer a fair, impartial and balanced investigation into the current capabilities of these systems, so that schools, teachers and students can better understand where and how software can play a role in supporting the instruction of writing.

Competition among participating vendors has no place in the Classroom Trials. The study is designed to identify commonalities across various product features and report the impact of those capabilities in supporting writing instruction.

As schools shift to online assessment over the next few years, computers will play a larger role in both how students are tested and how they are prepared for those tests. This presents a unique opportunity to inform school administrators, teachers and students of the potential to support new instructional demands.  At a time when the use of computers in classrooms is scheduled to change dramatically, we believe that offering a rigorous and unbiased assessment of tools that are specifically designed to support those new demands will benefit both providers and consumers of those services.

Finally, we are specifically interested in clarifying how software can support writing instruction.  We know that students are writing less today.  We also know that demands placed on teachers are a critical factor in addressing how much writing is assigned.  Put simply, as classroom sizes are growing, the level of detailed feedback provided to each student is falling.  Software can play a vital role to address those workload constraints, and teachers need more information to understand their potential.

How many products will be assigned to classrooms?

Based on the quality of the applicants, we will select between 3-5 products to place into classrooms.

In what classrooms will the products be placed?

We will work with our federal and state partners, as well as the selected vendors, to identify the specific classrooms for each product.  Because each product and provider varies, according to their ability to serve specific populations and even geographic regions, we are first committed to naming those products before matching them with specific classrooms.  We have designed the trials first to ensure that no product will present any threat to any specific student population.  Therefore, providers are asked to describe their track record serving students, and for those finalists who are invited to undertake this matching process we will determine whether specific students (e.g., ELL, ESL, Special Ed) should be subjected to those products.  We will work closely with the selected vendors to make those final determinations.

Another factor in determining which products should be placed in which classrooms is the required "intensity of the study or each individual intervention."  This will require us to have a more comprehensive understanding of the selected systems, before we can finalize classroom allocations for each product.  For some products, integration into standard or typical curricula is the objective.  For others, changes in curricula or specific training to integrate those products into classroom conditions are required. Therefore, we intend first to name the specific products and to determine their specific requirements for classroom integration before we can finalize the required "level of usage" and other study variables. We know that those variables are important to our partners, and we will work closely with the selected vendors, to ensure absolute transparency around how each system will be measured according to which classrooms are assigned.

What are the measurements that will be used in this study?

The study will focus on both qualitative and quantitative measures of usage and product performance.  Beginning with quantitative measures, we will determine which standard data can be uniformly extracted from the participating products before finalizing the common variables.  Where we intend to apply some common measure that one or more systems are not designed to generate, we will work with those providers to develop protocols that are fair or that would not corrupt any indication of specific use or performance.  Second, as we finalize the qualitative measures, we intend to employ surveys, semi-structured interviews and forums for discussion, to ensure that any naturalistic components of the study design are aligned with the specific capabilities of the selected products and the intended goals of the study (see STUDY DESIGN CONSIDERATIONS).  Because we are selecting specific products to implement in the study, we will invite those product providers into the final design considerations before releasing publicly specific metrics and protocols for gathering the data.

What is required to submit a product for consideration?

Specific application requirements are provided at this website.  Please visit STEP ONE at the home page to learn more.  Preconditions for applicants are provided there; they are:

1.  Applicants must submit evidence of prior performance; start-ups and early stage companies with no prior track record of supporting teachers and students are not invited to apply. 
2. For seasoned providers, the only indication of preferred product attributes are contained in the application, the trait scoring rubric used to assess the products, and anything that an Applicant can infer from detailed descriptions of those judges assigned to score the products - please read them carefully before applying.
3. While many products offer interesting methods for facilitating the instruction of writing, we are seeking products that offer one specific capability; some form of automated feedback to the student and/or teacher is necessary.  This precondition is an important consideration.  For many, it implies an application of "machine learning" in order to gauge the nature of the student's writing and to generate supportive feedback.  While this is a necessary component, it does not signify our only interest in the product capabilities.  In fact, while automated feedback is a precondition, other capabilities may be equally if not more essential to an overall strategy to support the instruction of writing in a classroom setting.
4. All Applicants must comply with the Terms & Conditions of the submission process.

If an Applicant accepts these preconditions and registers to apply, the requirements to complete the Application are described in Step One on the home page of this website.  The Application includes a two part process and a systems demonstration of 45-60 minutes (either to be scheduled or through a pre-recorded submission).  The first part of the Application is an Executive Summary of your product attributes, and the second part is a more detailed description of the product summary, where the Applicant can provide specific data to substantiate any claims made in the Executive Summary. 

How will our application be evaluated and scored?

Each of the Working Group members named on this website (see THE REVIEW PROCESS) will be responsible for reading and scoring all completed applications.  Those Judges will use the Trait Scoring Rubric provided on this website, to score and provide feedback against each of the three judging criteria for each Applicant.

How much time do we have to enter our submission?

Applications are due by Tuesday, May 14^th, 2013 (11:59pm, US EST).  This is a total open application period of approximately one month.  You will submit your Application using this website which remains available to you at all hours.  You may submit questions during regular working hours to:

[email protected]

When can we expect to hear whether our product is accepted into the Classroom Trials?

You can expect to hear approximately four weeks from the final deadline (May 13^th).  During this period, we may need to schedule product demonstrations, and those schedules depend upon the availability of our Working Group members.  If any need for more time arises during the assessment of the Applicants, each registered Applicants who completes the submission requirements will be informed.

Our product does not offer a score or other feedback to students; can our organization still participate in the Classroom Trials?

We are limiting the selected products only to those systems capable of offering some form of automated feedback to either the student or teacher.

We're excited about this opportunity, but we need a few more details. How can we get more information?

Please contact Lynn Van Deventer at [email protected]