EVALUATION OF COMMERCIALLY
AVAILABLE SOIL TEST-KITS
AN ASSESSMENT OF ACCURACY, EASE-OF-USE, COST AND TIME
Aafje van den Boogert, 12409413
Supervisor: Dr. B. JansenMentor: Dr. D.
Danesh
University of Amsterdam, 28-05-2021
Hein, T. (2017, June 13). Soil Testing Accuracy Important [Photograph]. Spud-Smart.Abstract
Soil testing is an important tool in minimizing negative environmental impact of fertilizers. It can identify soil nutrient deficiencies and stipulate fertilizer recommendations, which ultimately prevents harmful over-fertilization. Professional soil testing in a lab, however, is an expensive process, wherefore soil-testing is less accessible to small scale farmers and hobby-gardeners. Therefore, to increase soil testing, there is a need for high-quality commercially available soil kits. In this research three test-kits, LusterLeaf 1601 Rapitest, Lusterleaf 1663 soil test-kit, and Milwaukee MT6003 NPK soil test-kit, were evaluated on their accuracy, ease-of-use, cost and assessment time. Test-kit testing was conducted on clay-rich soil, sampled from a farm in Almere, the Netherlands. To assess accuracy of test-kit testing, soil samples were analyzed with both test-kit tests and lab tests, whereafter test-kit results and lab results were compared. Ease-of-use was assessed through a subjective analysis of the testing process of each studied test-kit. Furthermore, total cost per test-kit and cost per test-kit test were calculated and compared. Finally, each testing process was timed to eventually compare assessment time of each test-kit. Based on interviews conducted with hobby-gardeners, a professional-gardener and a farmer, accuracy, ease-of-use and cost are most probable to influence adoption-rate of test-kit testing. To test accuracy, test-kit results were compared with lab results with a Wilcoxon signed-rank test. Both Rapitest and LusterLeaf measured two nutrients (nitrate and potassium, and phosphate and potassium respectively) and pH-level accurately (p-value > 0.05, so no significant difference between the results from test-kit testing and lab testing), whereas Milwaukee only measured phosphate accurately, and did not test pH-level at all. Additionally, Rapitest was easiest to use due to its color-coded comparator and component. LusterLeaf was difficult to use due to its complex filtering device and the complexities of Milwaukee were experienced in the hazardous extraction solution. When comparing cost of the three test kits Rapitest is the cheapest in both total cost and cost per test. Lastly, even though Rapitest had a significantly longer assessment time than LusterLeaf and Milwaukee, this did not weigh as heavy in the test-kit evaluation as accuracy, ease-of-use and cost, because time was not identified as a barrier to test kit-use by the interview respondents. Therefore, Rapitest is the most effective and accessible test-kit.
Keywords: Soil testing, commercial soil test-kits, soil macronutrients, pH, motivations and barriers to test-kit testing
Table of Content
FOREWORD ... 4
INTRODUCTION ... 5
METHODS & MATERIALS ... 7
FIELD SAMPLING ...7
TEST-KITS ...8
LABORATORY TESTS ...8
STATISTICAL ANALYSES FOR ACCURACY AND TIME ...8
INTERVIEWS ...9
RESULTS... 10
RESULTS OF SOIL TEST-KITS ... 10
Accuracy of Rapitest ... 10
Accuracy of LusterLeaf ... 11
Accuracy of Milwaukee ... 13
Ease of use ... 15
Cost & Time ... 15
RESULTS OF INTERVIEWS ... 16
Hobby-gardeners’ motivations and barriers ... 16
Professional gardener’s barriers and motivations ... 19
Farmer’s motivations and barriers ... 20
DISCUSSION ... 21
TEST-KIT DISCUSSION... 21
Accuracy of test-kits ... 21
Motivation and barriers for test-kit use ... 22
Final comparison between test-kits... 22
RESEARCH LIMITATIONS AND FUTURE RESEARCH ... 23
CONCLUSION ... 25
DATA REPOSITORY ... 26
LITERATURE ... 27
ACKNOWLEDGEMENTS ... 30
APPENDICES ... 31
APPENDIX 1 – DESCRIPTION OF SOIL TESTING PROCESS OF EACH TEST-KIT ... 31
APPENDIX 2 – CONVERSION TABLES... 33
2A. LusterLeaf conversion table (LusterLeaf Products) ... 33
2B. Milwaukee conversion table (Milwaukee Electronics Kft.) ... 34
APPENDIX 3 – DATA ANALYSIS SCRIPTS ... 35
3A Accuracy of nitrate measured by Rapitest ... 35
3B Accuracy of phosphate measured by Rapitest ... 35
3C Accuracy of potassium measured by Rapitest ... 36
3D Accuracy of nitrate measured by LusterLeaf ... 36
3E Accuracy of phosphate measured by LusterLeaf ... 37
3F Accuracy of potassium measured by LusterLeaf ... 37
3G Accuracy of nitrate measured by Milwaukee ... 38
3H Accuracy of phosphate measured by Milwaukee ... 38
3I Accuracy of potassium measured by Milwaukee... 39
3J Accuracy of acidity (pH) measured by Rapitest and Milwaukee ... 39
3K Statistical comparison of time assessment of three test-kits ... 40
APPENDIX 4 – INTERVIEW GUIDE ... 41
APPENDIX 5 – TRANSCRIBED AND CODED INTERVIEWS ... 44
5A. Respondent 1 ... 44
5B. Respondent 2 ... 48
5D. Respondent 4 ... 56 5E. Respondent 5 ... 61 5F. Respondent 6 ... 63
Foreword
This research is part of the LEX4Bio project at the University of Amsterdam (UvA), which is part of the overarching LEX4Bio project as commissioned by the European Union. The LEX4Bio project at the UvA will assess the environmental impact of bio-based fertilizers, by studying organic contaminants and heavy metal content. This research will explore the possibility of soil testing with commercially available soil test-kits, which test nitrate, phosphate, potassium and pH content of soils. The results of this research could aim towards recommending a quality test-kit which could be used to deploy hobby-gardener to detect nutrient deficiency in soils. LEX4Bio could use these results to map soil nutrient deficiency and to identify trends of macronutrient- and pH-level in the soil. This information is valuable to identify where BBF’s are needed, and the effect of BBF’s could be tracked to some extent. This research on soil test kits is one of five bachelor thesis projects related to LEX4Bio project at UvA. Two other bachelor thesis projects will study the cost and efficiency of fertilization with sewage sludge compared to a combination of manure and mineral fertilizers on agricultural lands. The last two projects will investigate the potential heavy metal pollution from sewage sludge fertilizers as modelled with Visual MINTEQ and PHREEQC.
Introduction
To ensure food security for the ever-growing world population, global agricultural practices have intensified over the last decades and are predicted to continue to intensify to provide sufficient food (Xiang, Malik & Nielsen, 2020). While increasing food supply, the intensification of agriculture simultaneously increases pressure on soil fertility by depleting soil nutrient resources (Tilman et al., 2002). To counter depletion of soil fertility, the use of chemical fertilizers has increased (Scotti, Bonanomi, Scelza, Zoina & Rao, 2015), which consequently induces countable adverse environmental effects (Rengel et al., 2020). For example, leaching of nitrogen and phosphorus results in eutrophication and groundwater pollution, which poses a health hazards for both humans and animals consuming such contaminated waters (Beegle, Carton & Bailey, 2000; Tilman et al, 2001). Furthermore, excessive appliance of ammonium-based fertilizers induces soil acidification, which can lead to a reduction of cation exchange capacity and ultimately result in a decline in crop yields (Goulding, 2016). Additionally, the production of chemical fertilizers requires significant amounts of energy, which results in additional greenhouse gas emissions (Vaneeckhaute, Meers, Michels, Buysse & Tack, 2013). Moreover, phosphorus resources are finite and are estimated to be completely depleted within 75 years (Smit, Bindraban, Schröder, Conijn & Van der Meer, 2009). Since there is no replacement for phosphorus in plant growth, it is impossible to reach food security without these recourses (Rengel et al., 2020). The exhaustion of phosphorus reserves and the polluting effects of chemical fertilizers threaten the goal of providing food security for the growing world population. In order to sustainably reach this goal, a balance must be found between maintaining soil fertility to produce sufficient crops and minimizing negative environmental impact of fertilizers.
To achieve an agricultural system where soil fertility is maintained while diminishing adverse environmental effects induced by over-application of chemical fertilizer, a shift from traditional agricultural practices to more circular practices is needed (Xiong, 2014). Reusing biological rest-products, such as sewage sludge or residues from livestock production, attributes to achieving a more circular agricultural system (Chojnacka, Moustakas & Witek-Krowiak, 2020). These biological rest-products contain valuable nutrients, but they are often viewed as expensive, wasteful rest streams (Rengel et al., 2020; Riding et al., 2015). These nutrient rich rest streams (NRRSs) could be processed and recycled into bio-based fertilizers (BBF’s) so that valuable nutrients do not leave the agricultural system but are reused (Tur-Cardona et al., 2018). This way BBF’s contribute to the shift to a more circular agricultural system, which is both environmentally and economically beneficial (Vaneeckhaute et al., 2013). To further ensure circularity of the agricultural system and to minimize fertilizer impact such as eutrophication and soil acidification, unnecessary nutrient losses from BBF’s should be prevented (Beegle et al., 2000). This could be done by applying the correct amount of fertilizer corresponding to measured nutrient deficiencies of agricultural soils (Schröder, 2005). In assessing these soil nutrient deficiencies and in specifying fertilizer recommendations, soil testing is a valuable tool (Golicz, Hallet & Sakrabani, 2020).
Soil testing has an important role in soil fertility management (Head, Crockatt, Didarali, Woodward & Emmett, 2020). A generally accurate soil testing method is professional soil testing performed in a laboratory (lab testing). This soil testing method is often used by large-scale farmers and agronomists to assess soil fertility, compose fertilizer recommendations and ultimately increase yield (Head et al., 2020). However, despite its benefits for soil fertility, lab testing has not been adopted on a large-scale (Golicz, 2020). Low lab testing rates could be attributed to its high costs, wherefore small-scale cultivators, such as small-scale farmers or hobby-gardeners, or farmers in the global south might lack
the monetary recourses to access these services (Attanandana, Yost, & Verapattananirund, 2007; Faber, Downer, Holstege, & Mochizuki, 2007). Soil testing rate could be increased through the introduction of a low-cost alternate soil testing method (Golicz, 2020). Commercially available soil test-kits, which are a simplification of lab testing and can test soil macro nutrients and pH-levels on site (Attanandana, Yost, & Verapattananirund, 2007), form such an alternative. However, besides being a more accessible soil testing medium due to its low costs, accuracy of soil testing with test-kits (test-kit testing) has been a contentious issue which requires further assessment (Brandenberger, Browser, Zhang, Carrier & Payton; 2016). Additional factors that influence the adoption rate of test-kit testing include time and ease of use (Brandenberger et al., 2016; Head et al., 2020). However, by merely facilitating soil testing through identifying quality test-kits, a greater soil testing network is not guaranteed because a knowledge gap remains on motivation and barriers for people who cultivate crops (i.e., farmers and gardeners) to actually use test-kits. Therefore, research is needed not only to evaluate test-kits, but also to inquire the need and motivation of people who practice agriculture, to conduct test-kit testing.
By identifying quality soil test-kits, test-kit testing could become more accessible. Moreover, information on site-specific soil fertility could be increased by deploying citizens to conduct soil testing. In this aspect the relevance of this research lies; deploying citizens to contribute to data collection, also known as citizen science, has become an increasingly important data collection tool with multiple benefits over conventional data collection such as being less costly and less susceptible to misreporting for self-interest (Dobreva & Azzopardi, 2014; Fritz et al., 2019; Hsu, Malik, Johnson & Etsy, 2019). However, within soil testing, citizen science has not been widely applied as a data collection tool (Golicz et al., 2020). The results of this research contribute to the alignment between citizen and soil test-kits and can therefore be valuable for institutions seeking to increase the soil fertility data collection network through citizen science. By deploying citizens to contribute to soil fertility data, which is valuable in creating a sustainable agricultural system and ensuring food security (Head et al., 2020).
The aim of this research is to evaluate the extent to which commercially available test-kits can correctly assess soil macronutrients (N, P, K) and pH-level of the soil, and to compare the ease of use, cost and assessment time of three test-kits. Moreover, the aim is to gauge the motivation and barriers amongst people who cultivate crops to adopt test-kit testing. The latter will give insight into which test-kit factor (accuracy, ease of use, cost or time) influences adoption rate of test-kit testing the most. Lastly, this research aims to recommend the best soil test-kit option based on the tested factors and criteria set by people who cultivate crops.
To research the quality of soil test-kits the following main research question has been set up:
- How do different brands of soil test-kits compare in accuracy of macronutrients and pH assessment, ease of use, cost, and assessment time and which factors influence adoption-rate of soil testing with test-kits?
To assess accuracy of test-kit testing, and to gauge barriers and motivations of people who cultivate crops to perform test-kit testing, the following sub-questions have been set up respectively:
1. How do N, P, K and pH measurements from commercially available test-kits compare to laboratory measurements?
2. What are possible motivations and barriers amongst people who cultivate crops to conduct soil testing with soil test-kits?
It is hypothesized that soil test-kit accuracy, ease of use, cost and assessment time will vary amongst different test-kits (Brandenberger et al., 2016). The test-kit criteria that weigh heaviest are expected to be accuracy, which is important to increase soil health and yield, and time (Head et al., 2020; Sandhaus, 2017).
Methods & Materials
Field sampling
Clay rich soils were sampled from two cropping areas on a farm in Almere, the Netherlands (Figure 1) on 7th of April 2021, where cropping area A was grassland and on cropping area B potatoes were cultivated. Since this research is part of the overarching LEX4Bio project, samples were taken from two different cropping areas since two other thesis projects studied these two fields. Sampling from two different cropping areas has no further relevance for this research. Using a metal ogre, five samples were taken from each cropping area at a depth of 30 cm, and a pocketknife was used to deposit the soil into a plastic zip lock bag. If present, earthworms were removed from the soil, before soil was deposited into bags. Each sample bag was labelled accordingly to the randomly selected sampling location (Figure 1). Once the soil samples were collected, subsamples were placed in a separate, labelled zip lock bag to bring home, where test-kit testing would be conducted. The original sampling bags were brought to University of Amsterdam for lab testing.
Figure 1. Study area, a farm near Almere, the Netherlands, where soil samples were taken. Samples with ID A were taken on a grass field, samples with ID B were taken on a potato field.
Test-kits
The three test-kits that were compared in this study were: LusterLeaf 1601 Rapitest, by LusterLeaf products (hereafter ‘Rapitest’), Lusterleaf 1663 soil test-kit by LusterLeaf Products (hereafter ‘LusterLeaf’), and Milwaukee MT6003 NPK soil test-kit by Milwaukee Electronics Kft. (hereafter ‘Milwaukee’). These test-kits were selected based on availability on the internet. Each test-kit stated that it contained a nitrogen-test (N-test), phosphorus-test (P-test) and potassium-test (K-test). The compounds that were measured by each test kit were: nitrate (NO3-), phosphate (PO43-), and potassium cations (K+). Both Rapitest and LusterLeaf tested pH-levels as well, where Milwaukee did not. Rapitest contained 10 tests per nutrient and pH, LusterLeaf contained 20 tests per nutrient and pH and Milwaukee contained 25 tests per nutrient.
Tests for macronutrients and pH were performed in duplicate to assess the reliability of measured results (Cohen, Mylavarapu, Bogrekci, Lee, & Clark, 2007). Because of limited tests in Rapitest, five out of ten samples were randomly selected and tested in duplicate. When testing sample A03 and A12 with Rapitest, soil was not yet dried. Hereafter, soil samples were airdried for three days to minimize settling time. Tests were conducted according to test-kit instructions provided by manufacturers. Appendix 1 contains a brief description for each test-kit of the steps that were performed to conduct test-kit testing.
To compare the quality of the different test-kits, the accuracy of test-kit results was analysed which will be elaborated on in the paragraph on statistical analyses. Furthermore, the test-kits were used accordingly (Appendix 1), and notes were made regarding the benefits and difficulties of each test-kit conduction in order to subjectively assess ease of use. Total cost was used to calculate cost per test of each test-kit. Conduction process of each test was timed for later comparison.
Laboratory tests
The half of the soil of each soil sample was analysed in the laboratory of the University of Amsterdam. Firstly, 30 grams of soil for each soil sample were added to 75 mL of water, which was shaken for 30 minutes. Hereafter, pH of the resulting suspensions was measured with an electrode (Consort C831) (Miller & Kissel, 2010). After measuring the pH, samples were centrifuged for 20 minutes at a speed of 2000 rations per minute, to separate water from the soil. Hereafter, samples were filtered using a filter box. A membrane filter of 0.45 µm was used with a pressure of 0.18 atm. Filtrates were split in two, where duplicates were analyzed for nitrate and phosphate on a continuous flow autoanalyzer (Skalar San++ continuous flow analyzer) (Li, Zeng, Mao & Yu, 2012). Other duplicates of the filtrates were analyzed for potassium (emission line: 766,91 nm) using the ICP method (Optima-8000 ICP-OES, Perkin Elmer, Waltham, U.S.A.) (Hansen et al., 2013).
Statistical analyses for accuracy and time
To assess the accuracy of each test-kit, results of macronutrients and pH tested by test-kits were compared to laboratory results. It was assumed that soil lab testing is accurate, thus if test-kit test results coincide with lab results, kits were deemed accurate (Brandenberger et al, 2016). To compare test-kit results to laboratory results, all data was converted to ordinal data which fit the categories low (1), medium (2), high (3). Rapitest and LusterLeaf results were expressed in the categories depleted, deficient, adequate, sufficient and surplus. Depleted and deficient were categorized as low, adequate and sufficient were categorized as medium, and surplus was categorized as high. Milwaukee results were already expressed in low, medium, high, therefore this data did not need conversion. To convert
laboratory results from quantitative to ordinal, conversion tables were used (Appendix 1). Conversion tables were provided by LusterLeaf Products and Milwaukee Electronics Kft. Lab results expressed NO3- and PO43- in 𝜇mol/L and K+ in mg/L (ppm). NO3- and PO43—results were converted to ppm, using molar mass of 62,005 M and 94,971 M respectively. Hereafter, the lab results were converted to ordinal data. Since the conversion tables provided by LusterLeaf and Milwaukee differed, lab data was categorized accordingly. A Wilcoxon’s signed-rank test was conducted to compare the ordinal results of each test-kit pair wise with the ordinal lab results (Appendix 3) (McCrum-Gardener, 2008). A significance level of 0.05 was employed.
To assess the conduction time of each test-kit, conduction times were compared. Because Milwaukee does not test for pH, only test time of NPK testing was compared. Because the assumption of normally distributed data was violated (Shapiro-Wilk test: p < 0.05 (Shapiro, Wilk & Chen, 1968)) and because samples size n differed between test-kits (Rapitest: n = 10; LusterLeaf and Milwaukee: n = 20), a Kruskal-Wallis test was conducted to test whether the NPK conduction time differed significantly (Liu, 2015). A significance level of 0.05 was employed. As a multiple comparison post-hoc test, Dunn’s test was conducted, which was most suitable due to unequal sample sizes (Elliot & Hynan, 2011).
Interviews
To explore the motivations and barriers of people who cultivate crops to conduct test-kit testing, interviews were conducted. An interview guide was created to ensure plausibility and objectivity of the results (Appendix 4) (Kallio, Pietilä, Johnson & Kangasniemi, 2016). Questions were asked regarding fertilization regime to gauge respondents out-look on gardening/farming, and questions about possible use of test-kits gauged the motivations and barriers for people to perform test-kit testing. Lastly, questions were asked about willingness to participate in citizen science projects. To allow for follow-up questions and to leave space for elaboration by respondents, the interview was semi-structured (Kallio et al., 2016). Respondents included four hobby-gardeners, one professional gardener and one farmer. A brief description of socio-demographics (gender, age, residence) of each respondent has been provided in Appendix 5. Due to limited availability of respondents, respondents were recruited via a convenience sampling method (Acharya, Prakash, Sexena & Nigam, 2013). Due to the COVID-19 pandemic most interviews were conducted through video-calling. All respondents were native Dutch, so interviews were conducted and transcribed in Dutch and quotes were translated to English. Interviews were recorded and transcribed, whereafter codes were created and interviews were coded according to step 1 to 5 and step 7 to 10 of Burnard (1991) (Appendix 5).
Results
Results of soil test-kits
Accuracy of Rapitest
The comparisons between test-kit results and lab results to assess accuracy are presented in figures 2 to 6. In table 1 (p. 16), the p-values which were computed with the Wilcoxon signed-rank test when comparing test-kit results to lab results, are presented.
Firstly, to test Rapitest accuracy, macronutrient test results have been compared to lab testing results (Figure 2).
Figure 2. A comparison between macronutrients measured by lab testing and macronutrients measured by Rapitest test-kit testing. Macronutrients results are expressed in 1 = low, 2 = medium, 3 = high. Lab data was converted according to LusterLeaf Products conversion table (Appendix 2). Vertical lines in graph indicate in duplicate measured soil samples. Figure 2A compares nitrate results from both testing methods, Figure 2B compares phosphate results from both testing methods, and figure 2C compares potassium results from both testing methods.
Notably, in figure 2A lab measured nitrate and Rapitest measured nitrate coincide for all soil samples, except B09.2. Wilcoxon signed-rank test computed no significant difference between lab results and Rapitest results for nitrate measurements (Table 1). When comparing lab measured phosphate to Rapitest measured phosphate, a p-value below 0.05 was computed (Table 1), meaning a significant difference was detected between phosphate measured by the two testing methods. Rapitest reports medium phosphate concentrations for all soil samples, while lab testing reports low phosphate concentrations for all soil samples (Figure 2B). Potassium concentrations measured by Rapitest were higher for soil samples A03.1 & A03.2 and B02.1 & B02.2. All other soil samples tested the same as lab testing (Figure 2C). When statistically comparing potassium results with Wilcoxon signed-rank test no significant difference between the two test methods was computed (Table 1).
Figure 3. A comparison between acidity (pH) measured by lab testing and acidity (pH) measured by Rapitest test-kit testing. Vertical lines in graph indicate in duplicate measured soil samples. Rapitest test-kit testing measured pH in increments of 0.5.
Laboratory results reported a pH-level ranging between 7.53 and 7.69 (Figure 3). Rapitest measured a pH of 7.5 for all soil samples. Wilcoxon signed-rank test reported a significant difference between the pH results of test-kit testing and lab testing (Table 1).
Accuracy of LusterLeaf
The comparison between LusterLeaf measurements and lab measurements are presented in figure 4. Nitrate measurements from LusterLeaf coincide with less than half of the lab results. All nitrate results from LusterLeaf that differ from lab results underestimate the nitrate concentration (Figure 4A). The p-value that resulted from the comparison between LusterLeaf nitrate concentrations and lab nitrate concentrations was below the significance level of 0.05 (Table 1), meaning there is a significant difference between nitrate measured by the two testing methods. Phosphate measured by LusterLeaf coincided with lab measurements completely (Figure 4B), and p-value higher than 0.05 confirmed there was no significant difference between the testing methods (Table 1). Lastly, both LusterLeaf as lab-testing tested a low concentration of potassium for each soil sample and the corresponding p-value indicated no significant difference between the test results and the lab results (Table 1).
Figure 4. A comparison between macronutrients measured by lab testing and macronutrients measured by LusterLeaf test-kit testing. Macronutrients results are expressed in 1 = low, 2 = medium, 3 = high. Lab data was converted according to LusterLeaf Products conversion table (Appendix 2). Vertical lines in graph indicate in duplicate measured soil samples. Figure 4A compares nitrate results from both testing methods, Figure 4B compares phosphate results from both testing methods, and figure 4C compares potassium results from both testing methods.
Figure 5. A comparison between acidity (pH) measured by lab testing and acidity (pH) measured by LusterLeaf test-kit testing. Vertical lines in graph indicate in duplicate measured soil samples LusterLeaf test-kit testing measured pH in increments of 0.5.
Laboratory results reported a pH-level ranging between 7.47 and 7.69 (Figure 4). LusterLeaf measured a pH of 7.5 for all soil samples. When comparing testing methods with each other, a significant difference was computed with Wilcoxon’s signed-rank test (Table 1).
Accuracy of Milwaukee
Milwaukee test results have been compared to lab data (Figure 6) converted according to the conversion table provided by Milwaukee Electronics Kft. (Appendix 1). Nitrate measurements were higher when measured by lab-testing than when tested by Milwaukee for each soil sample (Figure 6A). A significant difference was computed using Wilcoxon’s signed-rank test (Table 1). When comparing phosphate measurements, measurements of soil samples A05.1 and B16.1 were higher when measured using the Milwaukee test-kit than when samples were tested in the lab. For soil samples A12.1 and B13.2 lab-testing measured higher phosphate concentrations than test-kit lab-testing (Figure 6B). A p-value higher than 0.05 indicates no significant difference between the results from the two testing methods (Table 2). Almost half of the potassium results of Milwaukee were higher than the potassium results of lab-testing (Figure 6C) and a significant difference was indicated (Table 1).
Figure 6. A comparison between macronutrients measured by lab testing and macronutrients measured by Milwaukee test-kit testing. Macronutrients results are expressed in 1 = low, 2 = medium, 3 = high. Lab data was converted according to Milwaukee Electronics Kft. conversion table (Appendix 2). Vertical lines in graph indicate in duplicate measured soil samples. Figure 4A compares nitrate results from both testing methods, Figure 4B compares phosphate results from both testing methods, and figure 4C compares potassium results from both testing methods.
Ease of use
In table 1, the benefits and complexities of the conduction process of each test-kit are presented. In the following sections, these benefits and complexities will be highlighted and explained.
The Rapitest test-kit was easy use due to colour coded reagents and comparators. Furthermore, each test only had one reagent, wherefore there was little room for error. However, because the comparators’ test chambers were small, and reagent capsule had to be opened with scissors, it was difficult to dispose all reagent into the test chamber without spillage, and cleaning the comparator thoroughly proved difficult. Additionally, because it was difficult to open reagent capsule without scissors, further spillage of reagents was unavoidable.
When conducting test-kit testing with LusterLeaf, merely complexities were noted. The use of the filtering device was complex, instructions had to be read multiple times, and practice was required before it could be used accordingly. Additionally, the filtering device had to be used for each test, and in between each test the filtering device had to be cleaned thoroughly. It was difficult to get the filtering device clean due to the small size of the different parts. This could have led to contamination between tested soil samples. Furthermore, the LusterLeaf test-kit had two to three different reagents per test, which had to be added at different stages in the testing process, wherefore there was more room for error.
The Milwaukee test-kit was easy to use because only one soil solution had to be made, which could be used for N, P and K test. Moreover, the test tubes in the Milwaukee test-kit were colour-coded, which assured using the same test-tube for each test, and additional contamination was prevented. Also, contamination was even more limited due to the brush, which made proper cleaning of the test-tubes easier. Different reagents were limited (one per test), which reduced possibilities for mistakes. However, soil solution was made with MT 5051 Extraction Solution, which contains the hazardous compound acetic acid. Acetic acid is highly corrosive to skin and eyes, and inhalation can cause damage to internal organs, so it had to be handled appropriately (Milwaukee Electronics Kft., 2019). The requested safety data sheet from Milwaukee Electronics Kft. indicated the need to wear protective clothes, gloves and goggles, which makes the Milwaukee test-kit less convenient to use.
Cost & Time
To compare test-kit cost, total cost was used to calculate cost per test (Table 1). Furthermore, the Kruskal-Wallis ANOVA on NPK conduction time, indicated a significant difference between the average conduction times. A post hoc Dunn-test computed a p-value below the significance level (p < 0.05) for all pair-wised compared test-kits, indicating a significant difference between all three test times averages. The average times are presented in table 1.
Table 1. A comparison is made between test kits based on accuracy, cost, ease of use and time. For accuracy: p-value < 0.05 indicates a significant difference between laboratory measurements and test-kit measurements, thus meaning low accuracy of test-kit. P-value > 0.05 indicates no significant difference between laboratory measurements and test-kit measurements, thus meaning high accuracy. For cost: All test-kits contained different number of tests. For Rapitest this was 10 tests for N, P, K and pH-test. For LusterLeaf this was 20 test for N, P, K and pH-test and for Milwaukee this was 25 tests for N, P and K-test. The latter did not contain any pH-tests.
Rapitest LusterLeaf Milwaukee Accuracy
Nitrogen-test p-value: 1.00 p-value: 0.00122 p-value: 2.15*10-5 Phosphate-test p-value: 0.0019 p-value:1.00 p-value: 1.00
Potassium-test p-value: 0.0719 p-value: 1.00 p-value: 0.00741
pH-test p-value: 0.00557 p-value: 0.011
-
Cost
Total cost € 17,32 € 42,95 € 53,75
Cost per test € 0,43 € 0,54 € 0,72
Ease of use
Benefits - Colour coded - Limited reagents
- Colour coded test tubes - Cleaning tool - Limited reagents Complexities - Inevitable reagent spillage
- Difficult to clean
- Complex filtering device - Difficult to clean - High number of reagents
- Health risk when improperly handled
Time
pH average 7,5 minutes 5 minutes -
NPK average 12 hours 29 minutes 22 minutes
Total average 12 hours 8 minutes 34 minutes 22 minutes
Results of interviews
Interviews were conducted to gain insight into the barriers and motivation of people who cultivate crops to perform soil testing with a test-kit. The main results for the interviews will be split up into results from hobby gardeners, results from professional gardener and results from farmers. The main motivations and barriers respondents perceived to using soil-test kits are depicted in figure 7.
Hobby-gardeners’ motivations and barriers
Four hobby gardeners were interviewed. The main motivation for them to use soil test-kits would be to improve garden health and gain knowledge on what additives the soil needs to be improved. As respondent 1 states: “It would of course be interesting to see what possibly has to be added (to my garden), to see what the soil is lacking and what would be a good idea to add. I see trees or bushes that are having a hard time, that do not grow as well as last year. Then I do not really know what is going on. Who knows, it might be nice if I could just see what is going on, very interesting”1. Furthermore, respondent 2 explains how “the grape-plant was doing really well the first four, five years, and last year it only gave one bunch of grapes. That is a sign that something is wrong with the soil, and then I decided to get rid of it and plant a new one (grape plant) in the following year. Maybe I should have used one
1
A translation of: “Zeker wel, omdat het natuurlijk interessant is om te zien wat er mogelijk aangevuld zou
moeten worden, wat er in de bodem ontbreekt en wat een goed idee zou zijn om toe te voegen. Ik zie natuurlijk ook wel bomen of struiken of planten die het moeilijk hebben, of die niet zo lekker gaan als vorig jaar. En dan weet ik ook niet precies wat er aan de hand is. En wie weet is het dan fijn als je gewoon kan zien wat er aan de hand is. zeker interessant.”of those test-kits, and then I would have seen what I should have added”2. Similarly, Respondent 3 explains that he would use the test-kits because then “I can base what I add on something more, and not just add something off the cuff.”3Respondent 4 is interested in improving her garden because she compares her garden to neighbours on the allotment complex and explains: “I am curious if there is a lack of something, because I see the differences between the gardens. In some gardens everything grows so fast. (…) I see large differences between gardens. So that makes me curious about the state of our garden.”4. She also states that “I have had this garden for many years, so a moment will come, which could be now, that I will think: Lets research the soil. For example, my strawberries do not grow at all. I’ve had them for a long time, so it might be because of that. But maybe it is the soil. (…) So that might be a reason (to use a soil kit); that I really want certain crops, and that I specifically ask myself: “what should I do for this crop?”. I could just buy new plants, but I could also research the soil first and see what it needs”5.
Another motivation for home gardeners to conduct soil testing is because of a sustainability perspective and because of an inherent, environmental care for the Earth (Figure 7). When asked questions about fertilization regime most respondents worked their garden in a sustainable manner. For example, respondent 3 says: “When I go to Groningen or Friesland, I see green asphalt, so a soil that is completely dead, that is mown three times a year. Insect life under the soil is totally disrupted in the Netherlands due to intensive use of machinery in agriculture (…) I try to leave the soil alone a little bit”6. None of the respondents use inorganic fertilizer because they do not deem it is necessary and prefer to use organic fertilizer if it grows crops. When asked about using soil test-kits in citizen science projects most respondents mentioned the importance of sustainability as well. For example, respondent 3 explains that: “if I was asked to provide data (of soil test-kit results) I would definitely do that. Unless it would be used to promote artificial fertilizer use (…) I would like to know that my data is used for ecological improvement of the soil”7. Respondent 4 states she would be open to using soil test-kits in a citizen science project because: “it serves a goal. I think that we all have to take care of the earth and we have to make sure it is going well (…) I want to commit to a societal goal, and for the environment and nature” 8.
2
A translation of: “zo'n druif die deed het heel goed, de eerste vier vijf jaar, en vorig jaar heeft die maar een
trosje gegeven. En dat is een teken dat er iets niet goed zit in die grond, dan denk ik ja, hup eruit, volgend jaar een nieuwe. Dan had ik misschien zo'n test moeten doen, en dat ik dan kijk wat ik toe moet voegen”.3
A translation of: “ik dan die toevoegingen iets meer kan baseren en niet zo uit de losse pols maar wat doen.”
4A translation of: “Maar ik ben wel benieuwd of er nou ergens een gebrek aan is, want je ziet wel verschillen
tussen tuinen. In sommige tuinen groeit gewoon alles als een tierelier. (…) ik zie wel bij andere tuinen grote verschillen. Dus dan ben ik wel benieuwd hoe dat bij ons zit. Ook omdat we heel lang op dit stukje zitten.” 5A translation of: De moestuin die heb ik natuurlijk al jaren, dus dan komt er een keer een moment, en dat zou best nu kunnen zijn, dat ik denk: laat ik het is wat beter onderzoeken. Bijvoorbeeld mijn aardbeien die doen het helemaal niet, maar die planten staan er ook wel heel erg lang, dus misschien ligt het daar wel aan. Maar misschien is het ook wel die bodem. (…) Dus dat zou dan wel een reden zijn. Dat ik bepaalde gewassen heel graag zou willen in de tuin, en dat ik mij dan echt specifiek af vraag: wat moet ik daarvoor doen? Ik kan ook wel nieuwe plantjes kopen, of zaaien, maar ik kan ook eerst even de bodem onderzoeken om te kijken wat nodig is.” 6
A translations of: “Als ik naar Groningen of Friesland ga dan zie ik groen asfalt, dus een bodem die totaal dood is, die drie keer per jaar gemaaid wordt. Het hele insecten leven onder de grond is helemaal verstoord in Nederland door intensief gebruik, mechanische landbouw, (…) Dus ik probeer de grond een beetje met rust te laten”
7: A translation of: “ als ik zou worden gevraagd of ik die gegevens door zou willen geven zou ik dat zeker willen doen. Tenzij ik dan zou merken dat dat gebruikt wordt om kunstmestgebruik te promoten. (…) Dus ik wil wel weten dat ze iets doen wat gericht is op een ecologische verbetering van de bodem.”
8
A translation of: “Ja, het dient gewoon een doel. Ik vind dat je ze zorgt met z'n allen voor deze Aarde en we moeten zorgen dat dat goed gaat. Stel dat er uit het onderzoek iets uitkomt de grond helemaal vervuild is, dat
Figure 7. The main motivations and barriers for hobby gardeners, professional gardeners and farmers to use soil test-kits.
The most prominent perceived barrier by hobby gardeners to use test-kits is high cost (Figure 7). Respondent 1 states ‘’if it is really costly, that would be a barrier as well”9. Respondent 2 explains that within citizen sciences projects “if the municipality would ask me to test the soil and fill in that data somewhere and they would provide test-kits for free”10 he would likely participate. Respondent 3 claims that the price would be a barrier for her when she says “if I would have to pay more than 40 or 50 euros for a test-kit, than I’d rather buy plants or other materials from that money”11. Respondent 4 agrees: “If a test-kit costs 100 euros, then I won’t do it (soil testing). You know, a test-kit of maximum of 25 euros. Thereafter I would not do it anymore, than I would rather just try something. Then I would buy seeds and try”12. Notably, respondent 2 explained that soil testing would potentially save money: “it (soil testing) prevents impulse purchases, that I would just buy some fertilizer or something pretty that I see in the gardening center and that I just guess: I think my soil needs this. It might save a lot of money”13.
vind ik dat een belangrijke informatie. Dus ik wil wel inzetten voor een maatschappelijk doel, en voor het milieu en natuur”.
9 A translation of: “En als het niet heel kostbaar is, want dat is ook wel een barrière”.
10 A translation of: “stel de gemeente vraagt mij om mijn bodem te testen en om dat een keer per maand ergens in te voeren en stellen die kits gratis ter beschikking”
11 A translation of: “Ja dus wel de prijs, als ik meer dan 40/50 euro zou moeten betalen voor die test dan koop ik daar liever plantjes voor of ander materiaal wat ik nodig heb.”
12 A translation of: “Als zo'n test-kit 100 euro kost, dan doe ik niet. Weet je, een test van max zo'n 25 euro. Daarna zou ik het dus niet meer doen, want dan ga ik liever iets proberen, dan koop ik zaadjes, en dan ga je proberen. Daar zou voor mij ongeveer de grens liggen”.
13 A translation of: “Het voorkomt ook impulsaankopen, dat ik dan toch maar nog iets van mest ga kopen, of iets anders moois wat ik in het tuincentrum zie en dat is dan maar ook de gok van: ik denk dat de bodem dit nodig heeft. Het kan misschien heel veel geld besparen.”
Another barrier in the use of soil test-kits by hobby gardeners was a lack of awareness of the existence of soil test-kits and awareness of what soil-testing entails and can do (Figure 7). Respondent 1, 2 and 3 had never heard of soil test kits. Respondent 3 had heard of laboratory soil testing but thought this to be the same as test-kit soil testing. Therefore, he deemed soil testing to not be useful in his garden because: “My garden is split into different plots that I have fertilized differently over the years. And with those tests they say, “take an average of the soil in your garden”. I know that that is not of much use for me because to some parts I have added soil improvement, and another part less (…) than a soil test won’t get me any further and I think: never mind’’14. Respondent 4 states something similar: “Because it is a really large part (of soil) (…) so we thought: where are we supposed to test exactly and do the results then apply everywhere?"15. This shows unawareness of what test-kits can do, wherefore the respondents chose not to use them.
Lastly, some respondents stated complexity to be a barrier to test-kit use. Respondent 1 stated about soil testing: “it should not be too complicated”16 and “I think it would be nice if it is simple to use”17. Respondent 2 explains that “If it is easy to use, then I will use it. It lowers the bar a little”18.
Professional gardener’s barriers and motivations
Respondent 6, a professional gardener, thought soil testing with test-kits could save him time and cost, which were both motivations for him to potentially use test-kits in the future. Regarding cost, respondent 6 explained: “it sounds interesting of course. (…) This way you can fertilize as efficient as possible. If you plant a lot of plants and a lot of those fall out, that is a waste for the customer. You don’t want the customer to have to pay more for plants”19. He also said that soil test-kits might be interesting to use on “smaller gardens, because for the bigger gardens soil is tested, but it costs some money to send it to the lab. On a bigger garden this is a smaller fraction of the budget. For a smaller garden is it a little out of proportion to test for so much money, so then a test-kit could be interesting”20. About time, respondent 6 says: “maybe it goes quicker. I don’t really know how long it takes (lab testing), but I think a few days. (…) First you need get soil, then you need to send it and then it has to come back, and you have to look at it. Yeah if (soil testing with a test-kit) is quicker than that might be handy, just so you have an indication”21. A barrier to his use is that respondent 6 did not know of the existence of soil-test kits. 14A translation of: “omdat de tuin verdeeld is in verschillende vlakken die ik over de jaren heen op
verschillende manieren heb bemest. En wat ze bij die tests zeggen, neem een gemiddelde van de grond in de tuin, en ik weet dat ik daar niet zo veel aan heb want op sommige plekken heb ik grondverbeteraar gegooid, een ander deel juist minder en daar heb juist veel bemest. Onder het gras ligt nog de originele grond, en dat is ook weer opgedeeld in twee delen, het ene deel daarvan is bemest en het andere deel niet. Dus dan schiet ik niks met zo'n bodem test op. Dan denk ik laat maar”
15 A translation of: “omdat het een heel groot stuk (…) en toen dachten we: ja, waar moeten we dat dan precies doen en geldt dat dan overal en eigenlijk weten we toch wel een beetje”.
16A translation of: ”het moet dan niet te ingewikkeld zijn”.
17 A translation of: “Ik denk dat het fijn zou zijn als het eenvoudig zou zijn om te gebruiken”.
18 A translation of: “JA als het its is wat heel makkelijk is, dan ga ik he gebruiken, dan ligt de lat wat lager”. 19 A translation of: “Dus op zich klinkt dat wel interessant natuurlijk. (…) Om het zo efficient mogelijk te bemesten. En ook als je heel veel planten gaat zetten, en er is heel veel uitval dat is natuurlijk zonde voor de klant want je wil ook niet de klant meer laten betalen voor de planten”.
20
A transaltion of: “kleinere tuinen, want bij een grotere tuinen laten we wel de bodem testen, het kost wel wat
geld om naar het lab te laten sturen. Maar als het een grote tuin is, dan is dat een heel klein bedrag. Als het een kleinere tuin is dan is het toch ook een beetje uit verhouding om dat voor zo veel geld te laten testen, zeg maar, dus dan denk ik dat een test-kit wel interessant kan zijn.”21
A translation of: misschien tijd ofzo, misschien dat het sneller gaat. Ik weet eerlijk gezegd ook niet hoelang
dat duurt (lab testen), maar ik denk een paar dagen. Ja dat moet opgestuurd worden, je moet eerst aarde halen, opsturen en dat moet dan weer een keer terugkomen en dan moet je het bekijken. Ja als het sneller is, ik denk dat dat sowieso wel handig is, als je in ieder geval een indicatie hebtFarmer’s motivations and barriers
Respondent 5, a farmer, did not see any motivation for him to use soil test-kit. He is already obligated by agricultural legislations to test his soil professionally once every four years. He explained that because soil testing with test-kits is less accurate than the laboratory soil testing he already does, he does not see any added values in the use of test-kits. He also states how time is a valuable product for him, and he would not have enough time to test the soil himself. Furthermore, since he already tests his soil in the laboratory, it would only lead to unnecessary additional costs to buy test-kits. Therefore, respondent 5 only saw barriers in using soil test-kits, as can be seen in figure 6.
Discussion
Test-kit discussion
Accuracy of test-kits
To assess the accuracy of test-kit testing, results from three different test-kits have been compared to lab testing results. Of the test-kits that were analysed in this study Rapitest and LusterLeaf tested for pH as well as macronutrients, where Milwaukee tested for macronutrients only. This following section will return to sub-question 1; how do N, P, K and pH measurements from commercially available test-kits compare to laboratory measurements?
Because no significant difference was detected when comparing Rapitest nitrate results to the laboratory nitrate results, it means that the Rapitest tested nitrate accurately. However, Rapitest measured medium concentrations of phosphate for each soil sample, whereas laboratory testing measured low phosphate concentrations per soil sample, meaning Rapitest overestimated phosphate concentrations. The computed p-value (Table 1) indicates that Rapitest does not measure phosphate accurately. Potassium measurements by Rapitest were measured correctly for 6 out of 10 soil samples. Rapitest overestimated potassium concentrations for all four soil samples of which the results did not correspond with lab testing results (Figure 2C). No significant difference between Rapitest and lab testing of potassium (Table 1), indicates that Rapitest accurately assesses potassium. When comparing Rapitest results of pH measurement to laboratory results, a p-value of below the significance level suggests that Rapitest assessment of pH is inaccurate. However, Rapitest only indicates pH-level in increments of 0.5 and tested a pH of 7.5 for each soil sample (Figure 3). If the results from the laboratory would be rounded to the nearest 0.5, all soil samples would also result in 7.5, wherefore pH testing by Rapitest is considered accurate.
When comparing nitrate test results from LusterLeaf to lab-testing, a p-value below the significance level (Table 1) indicates that LusterLeaf assesses nitrate generally inaccurately. When LusterLeaf results did not coincide with lab-testing results, LusterLeaf underestimated nitrate concentrations (Figure 4A). On the contrary, LusterLeaf tests phosphate the same as the laboratory test for each soil sample (Figure 4B). No significant difference is indicated here meaning the LusterLeaf tests phosphate accurately. At the same time, both LusterLeaf and laboratory testing assessed low levels potassium for all soil samples (Figure 4C), meaning LusterLeaf assessed potassium levels accurately. Lastly, for every soil sample LusterLeaf tested a pH for 7.5. With a p-value below the significance level (Table 1) the laboratory pH results were not assessed accurately. However, similar to Rapitest, LusterLeaf only expresses pH in increments of 0.5. When laboratory pH results would be rounded to the closest increment of 0.5, all pH values would be 7.5 thus coinciding with LusterLeaf results. Therefore, the LusterLeaf pH-test is deemed accurate.
Lastly, when comparing the Milwaukee nitrate results to the laboratory nitrate results, a p-value below the significance level indicated that Milwaukee assessed nitrate concentrations inaccurately. For four out of twenty soil samples, Milwaukee assessed the phosphate concentration differently than lab testing (Figure 6B). A p-value above the significance level indicates that Milwaukee assesses phosphate accurately. The potassium test of Milwaukee proves to be inaccurate (Table 1). Milwaukee generally overestimates potassium concentrations (Figure 6C).
To conclude, Rapitest is accurate in assessing nitrate, potassium and pH, but phosphate is generally overestimated. LusterLeaf is accurate in assessing phosphate, potassium and pH, but generally
underestimates nitrate. Lastly, Milwaukee accurately assesses phosphate, but nitrate is both overestimated and underestimated at times, and potassium is generally overestimated. The underestimation of nitrate by LusterLeaf and Milwaukee is similar to results from other studies (Burrows, 2004; Liebig, Doran & Gardner, 1996). The accurate assessment of pH level by both Rapitest and LusterLeaf corresponds to the results of other similar research (Brandenberger et al., 2016; Faber et al., 2007; Liebig, Doran & Gardner, 1996). As explained by Burrows (2004), test-kit results are accurate enough to return trends of macronutrients and pH of the soil.
Motivation and barriers for test-kit use
The following section will return to sub-question 2: What are possible motivations and barriers amongst people who cultivate crops to conduct soil testing with soil test-kits? When analyzing the motivations and barriers of people who cultivate crops to conduct test-kit testing of the soil, the focus will be on the results from gardeners. This is because most of the respondents were hobby-gardeners, making these results more significant than the results on professional gardeners’ or farmers’ motivations and barriers.
What became evident from the interviews conducted with hobby-gardeners was that the main motivations to use soil test-kits was to obtain more information of garden soil to ultimately improve yield or garden health (Figure 7). In order to be able to use test-kit results to improve garden health, the results have to be accurate, therefore this motivation relates to accuracy of test-kits. Secondly, hobby-gardeners were driven to use test-kits by sustainability (Figure 7). This perspective also relates to accuracy, because inaccuracy of test-kits leads to over-fertilization, which is accompanied by adverse environmental effects (Burrows, 2004; Rengel et al., 2020). This corresponds to findings from Beegle et al. (2000) and Sandhaus (2017), who explain that hobby-gardeners will likely be motivated to conduct soil testing because it improves gardening practices as well as environmental action. Barriers that became apparent when analyzing interviews, show that complexity of test-kit testing should be minimized for hobby-gardeners to adopt soil testing with test-kits, which corresponds with previous literature (Brandenberger et al., 2016; Head et al., 2020). However, previous literature does not highlight the barrier of cost in test-kit use, even though cost was identified as a barrier to test-kit testing by home-gardeners from the interviews (Figure 7). Nevertheless, in this study, motivation and barriers to test-kit use indicated that -kits must be accurate, low cost and easy to use, to increase adoption-rate of test-kits in soil testing.
Previous literature led to the hypothesis that high assessment time would negatively influence test-kit testing adoption-rate (Sandhaus, 2017). However, based on the interview results of hobby-gardeners, assessment time does not influence adoption-rate of test-kit testing.
Minimizing conduction time and complexity of testing will likely promote utilization of soil test-kits (Brandenberger et al., 2016; Head et al., 2020). While analyzing conducted interviews with citizens who practice agriculture, it became evident that complexity of test-kit conduction should be minimized for hobby-gardeners to adopt soil testing with test-kits. As discussed in the result section Milwaukee had the least difficulties in its use (Table 1). However, since the only difficulty experienced in the use of Milwaukee concerns potential risk to human health, this is considered a heavier weighing difficulty than for example
Final comparison between test-kits
To answer the main research question: How do different brands of soil test-kits compare in accuracy of macronutrient assessment and pH, ease of use, cost, and assessment time and which factors weigh
heaviest in adoption-rate?, a final comparison between the three test-kits will be made regarding accuracy, ease of use, cost and assessment time. The factors that weigh more heavily are accuracy, ease of use and cost. from interview data showed that these factors are important barriers for hobby gardeners to use soil test-kits.
In regard to accuracy, Rapitest and LusterLeaf were most accurate. Both test kits assessed two out of three nutrients accurately (nitrate and potassium, and phosphate and potassium respectively). Furthermore, they assess pH as accurately as possible with an increment scale of 0.5. Milwaukee only assessed phosphate accurately and was not able to measure pH at all. Accuracy could be lower due to limited color choices on the color comparison charts (Faber et al., 2007), where Rapitest and LusterLeaf had 5 color choices for nitrate and phosphate, and Milwaukee only 4. When comparing each test-kit on ease of use, Rapitest is relatively easy to use due to its color-coded comparators and reagents, and low number of reagents, wherefore the chances of adding the wrong reagent to the wrong test comparator are low. The most prominent disadvantage of Rapitest is the long settling time of the soil solution. Nevertheless, for hobby gardeners, this might not be an issue since one can leave the soil solution to settle unmanaged. LusterLeaf, on the other hand, has many different reagents and a complex-to-use filtering device, leading to much room for mistakes. Milwaukee has little complexities, however the possible health risk of the extraction solution in Milwaukee is a complexity that weighs heavy because it regards human health. When comparing cost, Rapitest has the lowest cost, whereafter LusterLeaf, whereafter Milwaukee with the highest cost (Table 1). In conclusion, Rapitest is recommended when testing soil nitrate, phosphate and potassium, and pH-level, due to its low cost, high ease-of-use and relatively accurate results.
Research limitations and future research
Within this research, five main limitations were encountered, which may have likely influenced the final results.
Firstly, the conversion table for Rapitest and LusterLeaf provided by LusterLeaf Products and the conversion table for Milwaukee provided by Milwaukee Electronics Kft. differed notable (Appendix 2). Margins of the LusterLeaf conversion table of potassium were much high than the Milwaukee conversion. To a lesser extent this is also the case for phosphate conversion. Herefore, nutrient concentrations might fall into a different category when converting lab results, which influences the accuracy assessment.
Secondly, reliability of the test-kit results is low because there is likely a difference between the interpretation of the colour match between the sample and the colour chart. Therefore, people might interpret results differently, which influences the reliability of test-kit results. For future research on the accuracy of test-kits it is recommended to have multiple people conduct test-kit testing independently (Faber et al., 2007). Furthermore, this study performed test-kit testing on one soil, which may influence accuracy (Faber et al., 2007). Follow-up research should use test-kits on different soils, to assess the capability of test-kit to accurately assess macronutrients and pH on different soils.
Thirdly, ease-of-use was evaluated based on subjective experience of conducting test-kit testing. To increase the significance of these results, it is recommended for ease-of-use to be assessed independently by multiple people performing test-kit testing.
To properly assess the quality of soil test-kits, precision of the test-kits is of importance (Golicz, 2020). However, due to time limitations, assessments of precision of these test kits were beyond the scope of this research. Variability of individual nutrient tests may be ascribed to the non-uniformity of the soil sub-sample (Burrows, 2004). It is expected that tests which test a small volume of soil, variability between duplicates might occur more often. Rapitest studies the largest volume of soil, when compared to LusterLeaf and Milwaukee, however, the effect of soil volume used in test-kit testing should be analysed in further research to draw definitive conclusions.
Another possible limitation, that may negatively influence reliability of this research, is the sampling method that was used for sampling of interview respondents. The sampling design of interview respondent was based on convenience sampling. The researcher’s own network was consulted for respondents. This might influence the results of interviews. For example, most people in researchers network have a high level of education. Since people who are highly educated are often more environmentally aware (Gillham, 2008), respondents might by more motivated to partake in test-kit testing to maintain soil fertility. Furthermore, due to time limitations, this research had a low number of respondents. To have a better understanding of the motivation and barriers of people who cultivate crops, a probability sampling method should be applied, and sampling size should be larger (Bleich & Pekkanen, 2013).
Conclusion
This research studied how three different brands of commercially available soil test-kits compared in accuracy of macronutrient and pH assessment, ease of use, cost and time assessment. Furthermore, the motivations and barriers of hobby-gardeners to use these soil-test kits was studied. When studying accuracy, Rapitest turned out to be the most accurate test-kit, correctly measuring nitrate, potassium and pH. Moreover, Rapitest is the most inexpensive test-kit option, costing € 17,32 in total and € 0,43 per test, where LusterLeaf costed € 42,95 in total and € 0,54 per test, and Milwaukee costed € 53,75 in total and € 0,72 per test. When assessing ease-of-use of each test kit, only for LusterLeaf complexities were noted. Milwaukee was easy to clean and there were few reagents, subsequently there were fewer possibilities for mistakes in the testing process. However, extraction solution that was used for Milwaukee test-kit testing forms a health hazard, which makes handling complex. Rapitest is difficult to clean wherefore contamination between soil samples is difficult to avoid, however, due to colour-coded comparators and reagents, Rapitest is easy to use. Lastly, all test-kits differ significantly in assessment time, where Milwaukee NPK-test time is quickest (average of 22 minutes), LusterLeaf NPK testing takes longer (average of 29 minutes), and LusterLeaf has a considerably longer NPK test-time, due to the settling time of soil solutions (average of 12 hours).
Based on conducted interviews with Dutch hobby-gardeners, the main criteria for a hobby-gardener to conduct test-kit testing was to improve garden health. Additionally, hobby-gardeners indicated that caring for the Earth through sustainable gardening practices was a motivation for gardeners to conduct test-kit testing. When employing test-kits to improve garden health and practice gardening in a sustainable manner, test-kits require to be as accurate as possible to avoid incorrect assessment of garden health and over-fertilization. Respondents indicated that if test-kits are expensive or if test-kit testing is difficult to apply correctly, this would impede the use of test-kits by hobby-gardeners.
Based on the outcome of the three compared test-kits, evaluated on accuracy, ease-of-use, cost and time, and based on motivations and barriers set by hobby-gardeners to use test-kits, Rapitest is recommended to increase test-kit use by people who cultivate crops. Rapitest is accurate, is the easiest to use due to colour-coded components and has the lowest cost of the three studied test-kits. The main disadvantage of Rapitest is its long assessment time of macronutrients. However, because assessment time has not been indicated as a barrier to test-kit use by interviewed hobby-gardeners, the factor time does not weigh as heavy as the factors accuracy, ease-of-use and cost. Therefore, Rapitest is recommended as the most effective and accessible test-kit which can be used to increase adoption-rate of soil testing by small-scale cultivators such as hobby-gardeners.
Data repository
The original, raw data has been stored on GitHub and can be access via the following link: https://github.com/Aafje/EvaluationOfSoilTestKits/tree/main
Literature
Attanandana, T., Yost, R., & Verapattananirund, P. (2007). Empowering farmer leaders to acquire and practice site-specific nutrient management technology. Journal of Sustainable Agriculture, 30(1), 87-104.
Beegle, D. B., Carton, O. T., & Bailey, J. S. (2000). Nutrient management planning: justification, theory, practice. Journal of Environmental Quality, 29(1), 72-79.
Bleich, E., & Pekkanen, R. (2013). How to report interview data. Interview research in political science, 1, 84-105.
Brandenberger, L. P., Bowser, T. J., Zhang, H., Carrier, L. K., & Payton, M. E. (2016). Evaluation of Testing Kits for Routine Soil Analyses. Discourse Journal of Agriculture and Food Sciences, 4(1), 1-10.
Burnard, P. (1991). A method of analysing interview transcripts in qualitative research. Nurse education today, 11(6), 461-466.
Burrows, R. (2004). Accuracy of home soil test kits on South Dekota soils. In Proceedings of the South Dakota Academy of Science (Vol. 83, p. 65).
Chojnacka, K., Moustakas, K., & Witek-Krowiak, A. (2020). Bio-based fertilizers: A practical approach towards circular economy. Bioresource technology, 295, 122223.
Cohen, M., Mylavarapu, R. S., Bogrekci, I., Lee, W. S., & Clark, M. W. (2007). Reflectance spectroscopy for routine agronomic soil analyses. Soil Science, 172(6), 469-485.
Dobreva, M., & Azzopardi, D. (2014). Citizen science in the humanities: a promise for creativity.
Elliott, A. C., & Hynan, L. S. (2011). A SAS® macro implementation of a multiple comparison post hoc test for a Kruskal–Wallis analysis. Computer methods and programs in biomedicine, 102(1), 75-8
Faber, B. A., Downer, A. J., Holstege, D., & Mochizuki, M. J. (2007). Accuracy varies for commercially available soil test kits analyzing nitrate–nitrogen, phosphorus, potassium, and pH. HortTechnology, 17(3), 358-362.
Fritz, S., See, L., Carlson, T., Haklay, M. M., Oliver, J. L., Fraisl, D., ... & West, S. (2019). Citizen science and the United Nations sustainable development goals. Nature Sustainability, 2(10), 922-930.
Gillham, P. F. (2008). Participation in the environmental movement: Analysis of the European Union. International Sociology, 23(1), 67-93.
Golicz, K., Hallett, S., & Sakrabani, R. (2020). Novel procedure for testing of soil field test kits involving paper strips. Soil Use and Management.
Goulding, K. W. T. (2016). Soil acidification and the importance of liming agricultural soils with particular reference to the United Kingdom. Soil use and management, 32(3), 390-399.
Hansen, T. H., De Bang, T. C., Laursen, K. H., Pedas, P., Husted, S., & Schjoerring, J. K. (2013). Multielement plant tissue analysis using ICP spectrometry. In Plant Mineral Nutrients (pp. 121-141). Humana Press, Totowa, NJ.
