Welcome to the technical repository for my 2024/2025 SOC paper, this is where I keep all my scripts, scientific tests, algorithms, and graphing programs, let me walk you through how it works

Dataset

The documentation for the dataset structure can be found in DATASET.md, this is only interesting for the nerds

Distribution

This is probably the easiest part of this whole thing, it's basically just making charts and computing percentages, say you have 12 male and 15 female respondents, what is the distribution? It's quite simple, here:

(number of elements in a group) / (number of elements in the dataset)

So in this case, the distribution of male respondents would be (12) / (12 + 15) = ~44%, and female (15) / (12 + 15) = ~56%, now that we know this, we can make a pretty pie graph! The script that does all of this is distribution.py

Analysis and scientific tests

This is where stuff gets interesting, the script that does all the heavy lifting is analyze.py, then, you have the specified analysis scripts, like analyze_sex.py (which, surprisingly, only analyzes sex)

analyze_sex.py only picks out its data from the dataset and passes it down to analyze.py to do all the analyses, where the following things happen:

the received data is put into groups, each receiving an assigned letter (A, B, C, etc), groups of insufficient size are removed
if there are less than 2 groups, analysis aborts
Kruskal-Wallis test is performed and F and p values are received
if p is greater than 0.05, the difference between those groups is not statistically significant and analysis aborts
post-hoc Dunn test is performed and p values are saved
a result table is created, a comparison between each group is added as well as their rank-biserial correlation, difference in medians, difference in means, and post-hoc p value

Problem solved! If the difference is statistically insignificant, or we don't have sufficient data to perform a statistical test, the analysis aborts, otherwise, we get our F value, p value, and the result table, which could look something like this:

Skupina 1	Skupina 2	Veľkosť účinku	Rozdiel priemerov	Post-Hoc p-hodnota
A	B	0.0440	0.4198	0.0497
A	C	0.0399	0.2723	0.5239
B	C	-0.0084	-0.1475	0.3706

Graphing

Once the analysis is complete, successful or not, we can graph the data, we mostly use violin plots, which are quite easy to understand and interpret, it goes like this:

the window is split into four subplots, top left for average grade, top right for math grade, bottom left for slovak grade, and bottom right for english grade
all groups get added to each subplot as a violin plot, so for sex, each subplot would contain a violin plot for males and a violin plot for females
the F and p values get added to the top left corner of each subplot
the legend gets added to the top right corner of each subplot and axes are marked
each violin plot contains five pieces of valuable information:
1. the shaded background that shows the distribution of the data
2. the gray line that represents the data between the first and third quartile
3. the red mean line with the mean value on the left
4. the green median line with the median value on the right
5. the minimum and maximum bounds
labels get added to each violin plot

Quite complicated, right? A ton of data packed into one small image, which could look something like this:

It can be overwhelming to look at at first, but once you understand what's going on, it's quite intuitive, anyway, the function that does all this is also saved in analyze.py

Neural network

Ah! AI stuff! Well, it didn't work in the end because of the abysmal amount of data, but the structure and the training process is still here and can be looked at

The script that trains the neural network is train_nn.py (yes, I am very creative, I am aware), it uses the pytorch library to do all the math stuff that goes on behind the scenes, but the important part is the structure of the neural network, right here:

Of course, we have to use the .npy file format to load the data into our program, so how do we convert the .csv data provided by the Google Forms into a .npy? The answer lies in clean.py, but I'm not going to go into how it all works, it's just cleaning the data

The whole training thing is pretty complicated, so if you don't know anything about neural networks, just forget about it and attribute it to magic, but if you do, read through train_nn.py, it's a pretty clean and readable code