SoilSight: A Systems-Based Approach to Soil Organic Carbon Estimation

Video Script

I'm Felicity Banbury. Soil Organic Carbon, the percent of organic carbon in soil, is the world's largest terrestrial carbon sink and is crucial for agricultural productivity. However, inapplicable and expensive soil measurement techniques prevent farmers from being able to justify regenerative practices, like what you see behind me.

That's where my project, SoilSight, comes in. It's a machine learning model that uses reflectance to predict Soil Organic Carbon. It's accurate, applicable, and soon to be accessible as an API on SurveyStack. SoilSight demonstrates immense promise in catalyzing the carbon farming industry and ensuring our climate adaptability in the years to come.

See you at the Fair!

Introduction

Soil Organic Carbon (SOC) represents the world’s largest terrestrial carbon sink and is crucial for agricultural productivity through nutrient and water storage (Figure 1) (Spotorno, 2026). I first learned about SOC when my father showed me a graph of our family’s farm’s SOC (Figure 2). In 11 years, SOC had increased from 1% to 4%, where a 1% increase normally takes decades (Georgiou, 2022). What I first noticed about this graph, however, was the 2-3 year sampling gaps. My father explained that because of the $50 per sample cost and laborious process, he’d taken samples intermittently and only on fields where he felt productivity increase from targeted amendments would be worthwhile.

This is not just a family farm predicament. Globally, research efforts using traditional measurement techniques (Figure 3) have created scattered, incompatible, and inaccessible soil metadata. A critical need exists for farmers to receive timely, cost-effective feedback on management practices through SOC estimates, allowing them to apply needs-based fertilization instead of time-based.

My solution, SoilSight, a machine learning (ML) model, takes a novel approach to SOC estimation by leveraging reflectance (Figure 4).

SoilSight is a continuation of my 2025 CWSF project, SoilSense. I made several key advancements, namely: collecting my own data, allowing a localized scope with uniform parent material; implementing ML pipelines using multiple models and ensembles; evaluating accuracy holistically using MAE, RMSE, MAPE and Mean Difference; and deploying SoilSight as a SurveyStack API so farmers can take in-situ SOC measurements (Figure 5).

Based on a literature review, I selected the following design criteria (Figure 6).

I collected 26 data points of 10 reflectance measurements with SOC values for model training. Samples were taken to 15 cm depth, into A-horizon (Molina, 2024), across my farm. GPS coordinates were noted for future covariate analysis (pH, clay content, NDVI) and results repeatability. Samples were air-dried for 48 hours and sieved at 2 mm to remove large inorganic particulates. Reflectance measurements were taken with OurSci Reflectometer in consistent lighting with localized spectral calibration. Measurements were collected in SurveyStack application and uploaded to Jupyter Notebooks as a Pandas Dataframe (Figure 7).

The Loss On Ignition (LOI) procedure determines percent composition by mass of organic compounds through muffle furnace combustion (Hoogsteen, 2015). 1 g samples were ground, dried at 105°C for 30 minutes, massed with a shielded electronic balance (+/- 0.0001 g), oxidized at 550°C for 3 hours in a muffle furnace, and remassed (Figure 8). SOC was 58% of mass loss (Murphy et al., 2019).

Using this dataset and various libraries, I deployed SoilSight (Figure 9).

The individual wavelengths were scaled using StandardScaler, ensuring equal effect by regression coefficients. Logarithms and ratios features were created and Standard Normal Variate calculations applied (Mokere, 2026).

Various models (Figure 10) were selected and cross validated using LeaveOneOut Cross Validation (LOOCV), where every data point becomes a validation point (scikit-learn developers, 2026) with Mean Absolute Error (MAE) for scoring. Although it is computationally expensive, LOOCV provides models with most training data possible while producing almost unbiased estimators. GridSearchCV was implemented to automate hyperparameter tuning. Dummy regressors (Figure 10) were also trained to evaluate performance. Final results were calculated through cross validation. Best estimators were selected for data fitting, generating final predictions, and API application.

Initial Statistical Preprocessing

Statistical analysis of SoilSight’s dataset showed a lower minimum, higher maximum, and larger standard deviation values than OurSci’s available dataset, demonstrating a more challenging but representative dataset for the local soil. The SoilSight dataset also has 10x fewer data points than OurSci’s; the limited number due to the expense involved in confirming SOC values in the lab.

Individual wavelength correlations to SOC were tested via linear regression. The strongest relationship was with the 850 nm wavelength with an R2 of 0.656 (Figure 11). However, the strong negative nonlinear correlation demonstrated the need for ML to generate insightful predictions.

Demonstrating Actual Learning

The 18 models trained were compared with a DummyRegressor optimized Baseline, which predicts using simple strategies (scikit-learn developers, 2026), using MAE (Figure 12). All DummyRegressor strategies’ accuracies were evaluated with the Constant Regressor, with the output of 2.87, performed the best with a MAE of 1.70. Five model types outperformed the Constant Regressor, demonstrating effective relationship generalization and learning.

Accuracy

The accuracy of these models was assessed through Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Mean Difference (Figure 13). The Ensemble estimator (E–1) had the lowest MAE of 1.40. The KRR had the smallest mean difference of -0.08, demonstrating the least predictive bias. The GPR had the lowest RMSE of 1.76, showing it made the fewest large errors. Finally, the RF had the lowest MAPE of 37.22%. All models have a negative mean difference, illuminating that they are biased to underestimate SOC content. All model MAPEs are elevated as SOC measurements are near-zero actual values. For example, if the prediction is 1.00% and the actual is 2.00%, the MAPE is 50%, despite being only 1 unit off.

The Actual vs. Predicted graph (Figure 14) provides further insight into not just the different estimator’s errors but where the errors are present. All learning models had residuals below 1.00% SOC in the 1% to 4% range, demonstrating accurate predictions, however all estimators substantially underestimate SOC beyond that range with residuals above 2% SOC. While validating soil improvements at these high percentages is crucial, this graph demonstrates that SoilSight is accurate at agricultural field levels of SOC, fulfilling its main goal.

Applicability

However, accuracy alone does not make results applicable. To measure applicability, an Area Under the Curve (AUC) graph, plotting True Positive Rate (TPR) to False Positive Rate (FPR), was created. Predictions were split into two classes, where samples predicted correctly to be over or under 4% labeled true positives while inverse were labeled false positives. SoilSight has an AUC of 0.92, where AUC=1.00 is perfect classification (Figure 15). As soil amendments are recommended below 4% SOC, this statistical descriptor reinforces SoilSight’s in-field applicability to inform farmers of general soil health without traditional sampling costs.

Discussion

A Reflectance SOC Estimation (SoilSight) system was developed and evaluated: a novel system capable of providing accurate, applicable, and minimal cost per sample localized predictors with open source tools (Figure 16).

SoilSight is unique in approach to SOC estimation as it maximises farm-sized datasets to make accurate predictions in a lightweight application for a defined agricultural area to facilitate positive climate outcomes (Figure 17).

So far, SoilSight:

Standardizes input sample reflectances

Leverages biochemical predictive factors

Describes meaningful reflectance and SOC relationships

Produces field-level accurate estimates

Will be implemented as API on SurveyStack

Empowers farmers by providing immediate management feedback through localized SOC estimators

SoilSight demonstrates immense promise for improving reflectance based SOC estimation from personal gardens to global estimators (Figure 18), as it shrinks sampling cost from $50 per sample to paying $500 CAD once for a tool that facilitates unlimited SOC estimations. In a world where soil may become a carbon source instead of a sink due to rising atmospheric temperatures (Viscarra Rossel, 2024), humankind can drastically increase climate adaptability by developing localized predictors to facilitate SOC monitoring and sequestration.

What if ML gets it wrong?

Quantifying loss or gain of SOC is crucial, as overestimation leads to overconfidence in soil quality, whereas underestimation understates soil improvements (Figure 19). Even my best model made predictive errors due to limited dataset size and spectral noise (Peng, 2020). Improving model accuracy, ensuring consistent data collection, and continuing to leverage SOC chemical features to guide ML predictions can reduce errors.

Next steps include:​

Increasing SoilSight dataset to realize higher predictive accuracy, especially at higher SOC values, and allow R2 calculation through different cross validation.

Implementing SurveyStack API with user-friendly interface that acknowledges estimative error while providing immediate field-level accurate estimates (Figure 20). (I am currently working with OurSci, the company that makes the Reflectometers, to deploy SoilSight.)

Obtaining other localized datasets to build other localized models for in situ SOC estimation for farmers.

Further research includes evaluating different sampling techniques, mid infrared (MIR) reflectance in-situ prediction (Figure 21), other applications of reflectance modeling (Figure 22) and soil carbon cycling.

I would like to express my immense gratitude to (Figure 23):

Dr. Jalil Assoud and Tristan Licksai (University of Waterloo), for performing the LOI testing on the 26 samples for the SoilSight dataset;

Dr. Dan TerAvest (OurSci), for his invaluable guidance and discussion with the reflectometer set-up and expertise in the field;

Madame El Kibbi, for her continued support and encouragement throughout my science fair journey;

Ms. Epplett-Stuart, for being the Teacher Supervisor of CHCI’s Science Fair club, for her continued support in enabling students to participate in science fairs;

The WWSEF team, for their valuable assistance in preparation, continued support and contagious enthusiasm;

My parents, for their continued love and support

References

An, D. (2023, August). Non-intrusive soil carbon content quantification methods using machine learning algorithms: A comparison of microwave and millimeter wave radar sensors. KeAi, 2(3), 152-166. https://doi.org/10.1016/j.jai.2023.09.001

Apesteguia, M., Plante, A. F., & Virto, I. (2018, March). Methods assessment for organic and inorganic carbon quantification in calcareous soils of the Mediterranean region. Geoderma Regional, 12, 39-48. https://doi.org/10.1016/j.geodrs.2017.12.001

Baldock, J. A. (2007). Composition and Cycling of Organic Carbon in Soil. Springer Nature Link, 1-35. https://link.springer.com/book/10.1007/978-3-540-68027-7

Banbury, F. K. T. (2025). SoilSense: A Machine Learning Approach to Soil Organic Carbon Estimation. ProjectBoard. https://partner.projectboard.world/ysc/project/soilsense-a-machine-learning-approach-to-soil-organic-carbon-estimation

Cambou, A. (2022, May). Prediction of soil carbon and nitrogen contents using visible and near infrared diffuse reflectance spectroscopy in varying salt-affected soils in Sine Saloum (Senegal). CATENA, 212. https://doi.org/10.1016/j.catena.2022.106075

Chen, Z. (2025, August 25). A national soil organic carbon density dataset (2010–2024) in China. scientific data, 12. https://www.nature.com/articles/s41597-025-05863-3

Dadgar, M., & Faramarzi, S. E. (2024, November 22). Assessing the performance of machine learning models for predicting soil organic carbon variability across diverse landforms. Springer Nature Link, 83. https://link.springer.com/article/10.1007/s12665-024-11960-0#:~:text=Soil%20organic%20carbon%20(SOC)%20is,had%20SOC%20content%20below%201%25.

Ewing, P. M., TerAvest, D., Tu, X., & Snapp, S. S. (2021). Accessible, affordable, fine-scale estimates of soil carbon for sustainable management in sub-Saharan Africa. Soil Science Society of America Journal, 85(5), 1-13. https://doi.org/10.1002/saj2.20263

Farm Lab. (2026). Advancing Soil Carbon Quantification with Remote Sensing and Machine Learning. Farm Lab. Retrieved April 29, 2026, from https://getfarmlab.com/advancing-soil-carbon-quantification-with-remote-sensing-and-machine-learning/#:~:text=The%20model%20appears%20to%20perform,stratification%20and%20sampling/lab%20testing.

Feng, L., Chen, S., Chu, H., & Zhang, C. (September, 15). Machine-learning-facilitated prediction of heavy metal contamination in distiller's dried grains with solubles. Environmental Pollution, 333. https://doi.org/10.1016/j.envpol.2023.122043

Geladi, P., & Kowalski, B. R. (1986). Partial least-squares regression: a tutorial. Analytica Chimica Acta, 185, 1-17. https://doi.org/10.1016/0003-2670(86)80028-9

George, K. J., Kumar, S., & Raj, R. A. (2020, July 25). Soil organic carbon prediction using visible–near infrared reflectance spectroscopy employing artificial neural network modelling. Current Science, 119(2), 377-381. jstor. https://www.jstor.org/stable/27229874

Georgiou, K. (2022, July 1). Global stocks and capacity of mineral-associated soil organic carbon. nature communications, 13. https://www.nature.com/articles/s41467-022-31540-9

Government of Canada. (2024, October 17). Soil organic matter. Indicators. Retrieved April 27, 2026, from https://agriculture.canada.ca/en/environment/resource-management/indicators/soil-organic-matter

Government of Canada; Agriculture and Agri-Food Canada; Science and Technology Branch. (n.d.). Soil organic carbon (%) - Soil Landscape Grids of Canada, 100m. Government of Canada. Retrieved April 27, 2026, from https://open.canada.ca/data/en/dataset/c9534015-e8b6-4e09-b2eb-2c9f8649208e

Hau, N.-X. (2024, December 15). Estimation of soil organic carbon content using visible and near-infrared spectroscopy in the Red River Delta, Vietnam. Chemometrics and Intelligent Laboratory Systems, 255(15). https://doi.org/10.1016/j.chemolab.2024.105253

He, T., Wang, J., Lin, Z., & Cheng, Y. (2009, January 27). Spectral features of soil organic matter. Geo-spatial Information Science, 12, 33-40. 10.1007/s11806-009-0160-x

Hoogsteen, M. J. J. (2015, February). Estimating soil organic carbon through loss on ignition: Effects of ignition conditions and structural water loss. European Journal of Soil Science, 66(2). 10.1111/ejss.12224

Hutengs, C. (2019, December 1). In situ and laboratory soil spectroscopy with portable visible-to-near-infrared and mid-infrared instruments for the assessment of organic carbon in soils. Geoderma, 355(1). https://doi.org/10.1016/j.geoderma.2019.113900

ISRIC. (n.d.). Data and Resources. ISRIC. Retrieved April 28, 2026, from https://isric.org/explore/

Kögel-Knabner, I. (2002, February). The macromolecular organic composition of plant and microbial residues as inputs to soil organic matter. Soil Biology and Biochemistry, 34(2), 139-162. https://doi.org/10.1016/S0038-0717(01)00158-4

Kos, J., Pavelek, D., & Kaykhaii, M. (2025, September 1). Unveiling the transformative power of near-infrared spectroscopy in biomedical and pharmaceutical analysis: Trends, advancements, and applications. European Journal of Pharmaceutical Sciences, 212. https://doi.org/10.1016/j.ejps.2025.107175

Lai, Y.-Q., Wang, H.-L., & Sun, X.-L. (2021, July). A comparison of importance of modelling method and sample size for mapping soil organic matter in Guangdong, China. Ecological Indicators, 126. https://doi.org/10.1016/j.ecolind.2021.107618

Losada, M. (2023, March 17). Mammal and tree diversity accumulate different types of soil organic matter in the northern Amazon. iScience, 26(3). https://doi.org/10.1016/j.isci.2023.106088

McBratney, A. B. (2003, November). On digital soil mapping. Geoderma, 117(1-2), 3-52. https://doi.org/10.1016/S0016-7061(03)00223-4

Meena, R. S. (2024). Chapter 10 - Significance of soil organic carbon for regenerative agriculture and ecosystem services. Biodiversity and Bioeconomy, 217-240. https://doi.org/10.1016/B978-0-323-95482-2.00010-9

Mettler Toledo. (2026). Mettler Toledo® MX205DU 30665094 Semi-micro Analytical Balance 82 g x 0.01 mg and 220 g x 0.1 mg. Scales Galore. Retrieved April 29, 2026, from https://www.scalesgalore.com/product/Mettler-Toledo-MX205DU-30665094-Semimicro-Analytical-Balance-82-g-x-001-mg-and-220-g-x-01-mg-px63211c34.cfm?srsltid=AfmBOorTTiaP_PA86nGNlgUSQI01TscJnlT7v1Wza2yQTP7mlAV07o_wZzc

Mokere, R. (2026, January 27). Soil spectroscopy improves mid infrared soil property prediction through optimized preprocessing and variable selection. Frontiers in Soil Science Pedometrics, 6. https://doi.org/10.3389/fsoil.2026.1760011

Molina, J. A. (2024, February). Soil depth and vegetation type influence ecosystem functions in urban greenspaces. Applied Soil Ecology, 194. https://doi.org/10.1016/j.apsoil.2023.105209

Murphy, B. W., Wilson, B. R., & Koen, T. (2019, July 23). Mathematical Functions to Model the Depth Distribution of Soil Organic Carbon in a Range of Soils from New South Wales, Australia under Different Land Uses. soil systems, 3(3). https://doi.org/10.3390/soilsystems3030046

Nepal, J., & Xin, X. (2025, July 23). Understanding soil carbon: Key ingredient to build healthy soils. Science Societies. Retrieved April 27, 2027, from https://www.sciencesocieties.org/publications/csa-news/2025/june/understanding-soil-carbon-key-ingredient-to-build-healthy-soils?q=publications%2Fcsa-news%2F2025%2Fjune%2Funderstanding-soil-carbon-key-ingredient-to-build-healthy-soils%2F

Oliveira, M. M., Miliao, G. L., & Rahman, M. (2026, June). Potential of NIR hyperspectral imaging for fast fraud detection in minced beef products with oat flour and cornstarch. Applied Food Research, 6(1). https://doi.org/10.1016/j.afres.2026.101726

OurSci. (2026). Reflectometer Overview. OurSci. Retrieved April 30, 2026, from https://www.our-sci.net/reflectometer/#:~:text=The%20Our%20Sci%20Reflectometer%20is%20a%20handheld,*%20Quick%20Carbon%20Reflectometer%20*%20Bionutrient%20Meter

Peng, L. (2020, June 12). Comparisons of the prediction results of soil properties based on fuzzy c-means clustering and expert knowledge from laboratory Visible – Near-Infrared reflectance spectroscopy data. Canadian Journal of Soil Science. https://doi.org/10.1139/cjss-2020-0025

Ribeiro, S. G. (2021, November 24). Soil Organic Carbon Content Prediction Using Soil-Reflected Spectra: A Comparison of Two Regression Methods. remote sensing, 13(23). https://doi.org/10.3390/rs13234752

Schreiber, B. (2005, September). Adsorption of dissolved organic matter onto activated carbon—the influence of temperature, absorption wavelength, and molecular size. Water Research, 39(15), 3449-3456. https://doi.org/10.1016/j.watres.2005.05.050

scikit-learn. (n.d.). scikit-learn: machine learning in Python — scikit-learn 1.8.0 documentation.

scikit-learn: machine learning in Python — scikit-learn 1.8.0 documentation. Retrieved April 27, 2026, from https://scikit-learn.org/stable/

scikit-learn developers. (2026). LeaveOneOut. scikit learn. Retrieved April 29, 2026, from https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.LeaveOneOut.html

SoilGrids. (2026). SoilGrids250m 2.0. Retrieved April 28, 2026, from https://soilgrids.org/

Soil Quality. (2024, July). Soil Organic Carbon Testing. Soil Quality Knowledge Base. Retrieved April 27, 2026, from https://soilqualityknowledgebase.org.au/measuring-soil-organic-carbon/

Spotorno, S. (2026, April). From soil carbon towards system sustainability: Integrating SOC modelling and life cycle assessment to evaluate environmental trade-offs in carbon farming.

Farming System, 4(2). https://doi.org/10.1016/j.farsys.2025.100195

University of Alberta. (2026). List of Services and Pricing. University of Alberta. Retrieved April 27, 2026, from https://nral.ualberta.ca/prices/

University of Massachuetts. (2025, April 25). Lab Services : Soil and Plant Nutrient Testing Laboratory Services : Center for Agriculture, Food, and the Environment (CAFE) at UMass Amherst. UMass Amherst. Retrieved April 29, 2026, from https://www.umass.edu/agriculture-food-environment/services/soil-plant-nutrient-testing-laboratory/lab-services

Viscarra Rossel, R. A. (2006, March). Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties. Geoderma, 131(1-2). https://doi.org/10.1016/j.geoderma.2005.03.007

Viscarra Rossel, R. A. (2024, March 26). A warming climate will make Australian soil a net emitter of atmospheric CO2. climate and atmospheric science, 7(79). https://www.nature.com/articles/s41612-024-00619-z

vishakha_1. (2008, April). sensor to measure organic carbon content from soil. arduino.cc. Retrieved April 27, 2026, from https://forum.arduino.cc/t/sensor-to-measure-organic-carbon-content-from-soil/521210

Yang, C. (2025, August 1). Spatio-temporal mapping reveals changes in soil organic carbon stocks across the contiguous United States since 1955. Communications Earth & Environment, 6. https://www.nature.com/articles/s43247-025-02605-6

All images were sourced on Canva except the cow photos (Thanks, References, and in Figure 2) which were taken by my father.