How to Conduct a Controlled Experiment in Software Engineering

This article explains how the Latin Square Design experiment was conducted within an academic context.


This content originally appeared on HackerNoon and was authored by Pair Programming AI Agent

Abstract and 1. Introduction

2. Experiment Definition

3. Experiment Design and Conduct

3.1 Latin Square Designs

3.2 Subjects, Tasks and Objects

3.3 Conduct

3.4 Measures

4. Data Analysis

4.1 Model Assumptions

4.2 Analysis of Variance (ANOVA)

4.3 Treatment Comparisons

4.4 Effect Size and Power Analysis

5. Experiment Limitations and 5.1 Threats to the Conclusion Validity

5.2 Threats to Internal Validity

5.3 Threats to Construct Validity

5.4 Threats to External Validity

6. Discussion and 6.1 Duration

6.2 Effort

7. Conclusions and Further Work, and References

3.1 Latin Square Designs

The main features of Latin square designs are that there are two blocking factors. Each treatment is present at each level of the first blocking factor and is also present at each level of the second blocking factor. This design is arranged with an equal number of rows (factor one) and columns (factor two). Treatments are represented by Latin characters symbols where each symbol is present exactly once in each row, and exactly once in each column. An example of the arrangement of this design is shown in Table 1.

\ Table 1: Latin square design with three treatments

\ In a Latin square design, blocking is used to systematically isolate the undesired source of variation in the comparison among treatments. In this case, pair versus solo programming. As a teaching purpose, we decided to block treatments by program and by tool support. Table 2 shows the arrangement used for the experiment.

\ Table 2: Latin square design arrangement

\ The program block has two levels: Calculator an encoder whereas tool support block has the levels: IDE (Integrated Development Environment) and text editor. The treatments to examine are: Pair and solo programming.

\

:::info Authors:

(1) Omar S. Gómez, full time professor of Software Engineering at Mathematics Faculty of the Autonomous University of Yucatan (UADY);

(2) José L. Batún, full time professor of Statistics at Mathematics Faculty of the Autonomous University of Yucatan (UADY);

(3) Raúl A. Aguilar, Faculty of Mathematics, Autonomous University of Yucatan Merida, Yucatan 97119, Mexico.

:::


:::info This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by Pair Programming AI Agent


Print Share Comment Cite Upload Translate Updates
APA

Pair Programming AI Agent | Sciencx (2025-08-19T01:35:47+00:00) How to Conduct a Controlled Experiment in Software Engineering. Retrieved from https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/

MLA
" » How to Conduct a Controlled Experiment in Software Engineering." Pair Programming AI Agent | Sciencx - Tuesday August 19, 2025, https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/
HARVARD
Pair Programming AI Agent | Sciencx Tuesday August 19, 2025 » How to Conduct a Controlled Experiment in Software Engineering., viewed ,<https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/>
VANCOUVER
Pair Programming AI Agent | Sciencx - » How to Conduct a Controlled Experiment in Software Engineering. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/
CHICAGO
" » How to Conduct a Controlled Experiment in Software Engineering." Pair Programming AI Agent | Sciencx - Accessed . https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/
IEEE
" » How to Conduct a Controlled Experiment in Software Engineering." Pair Programming AI Agent | Sciencx [Online]. Available: https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/. [Accessed: ]
rf:citation
» How to Conduct a Controlled Experiment in Software Engineering | Pair Programming AI Agent | Sciencx | https://www.scien.cx/2025/08/19/how-to-conduct-a-controlled-experiment-in-software-engineering/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.