Soul with a capital

(1)

Soul with a capital “S”

Automatic classification of American soul labels from the 1960’s and 1970’s

Karen Beckers

10811958

Bachelor thesis

Credits: 18 EC

Bachelor Artificial Intelligence

University of Amsterdam

Faculty of Science

Science Park 904

1098 XH Amsterdam

Supervisor

Dr. J.A. Burgoyne

Institute for Logic, Language and Computation

Science Park 107

1098 XG Amsterdam

(2)

2

Abstract

This research aims to classify three American soul record labels between 1959 and 1983, Motown, Stax and Philadelphia International, based on music content and using automatic classification techniques. A dataset of 1859 audio samples are collected and 32 features within the domains dynamics, fluctuation, rhythm, timbre, pitch and tonal are extracted. Classification experiments are conducted by applying a Support Vector Machine, Random Forest, Gradient Boosting and Adaptive Boosting in combination with the feature selection techniques of a variance threshold, Recursive Feature Elimination (RFE) and Recursive Feature Elimination with Cross-Validation (RFECV). The overall best results (mean accuracy is 0.98 and standard deviation is 0.01) are obtained when Motown and Philadelphia International are to be distinguished by employing Adaptive Boosting with either no feature selection, RFE or RFECV. Eleven features are ranked as being most significant for this problem and can be interpreted by the musical characteristics of each label.

(3)

3

1 Introduction ... 4

2 Theoretic framework ... 5

2.1 Characteristics of record labels ... 5

2.1.1 Motown ... 5

2.1.2 Stax ... 6

2.1.3 Philadelphia International... 7

2.2 Genre classification ... 8

3 Method and approach ... 8

4 Results ... 10

5 Discussion ... 12

6 Conclusion ... 15

References ... 16

Discography ... 18

Appendices ... 19

A Hardware and software specifications ... 19

(4)

4

1 Introduction

In 1960 America the combination of jazz, blues and gospel led to the emergence of a new music genre: soul (Borthwick & Moy, 2004). During this rowdy period of segregation and discrimination of the African-American community, the Civil Rights Movement and its more radical successor the Black Power Movement arose. Both movements opposed the idea of black people having a lower position in the community. By organising non-violent protests, they advocated equal treatment of all people in society (Maultsby, 1983). Soul music was aimed towards the African-American population and became the voice of the Civil Rights Movement. Through the tapered and belittled attitude of the blues and gospel a new and emotional spirit evolved that stood for pride and making changes in society. On one hand, soul called on protests for equality and social change (Borthwick & Moy, 2004; Maultsby, 1983). On the other hand, it showed that people from different backgrounds could work together. Within the music, race was not an issue; musicians were joined through the music. This was visualised by many bands that consisted of mixed-race musicians and record labels with white owners and black artists. Soul proved that coming from a minority group did not matter and that all races were equal (Borthwick & Moy, 2004).

Within this genre, the song and the story it tells is the most important aspect of the music; the technique came second. The songs were carried by a lead-vocalist, supported by harmonies from the backing-vocalists, horns and sometimes strings (Borthwick & Moy, 2004). For the first time, aspects from typical “black” genres like gospel crossed over to the “white” charts through soul music. These are elements such as call and response, where the lead melody is “answered” by a successive phrase, and the stretching of a syllable over multiple notes, which is called a melisma (Borthwick & Moy, 2004; Fitzgerald, 2007). The proclamation by the Beatles and the Rolling Stones of copying these styles of singing led to the acceptance of the genre by a much wider audience (Maultsby, 1983).

Soul music was a genre that, more than other genres, was carried by its record labels. Besides the many great artists that established this style of music, the “sound” of soul was to a great extend constructed by the house bands of the different labels (Borthwick & Moy, 2004). Each label strived to create their own individual sound that would set them aside from the other labels (Gordon & Neville, 2007). This suggests that there was a lot of rivalry between the record labels, of which Tamla Motown was undeniably the most famous at that time. This label was so concerned with maintaining their specific sound that they even spied on their musicians to see if they were not recording for other record labels (Justman, Slutsky & Passman, 2002). The Detroit based company brought forward many great artists, such as Stevie Wonder, Marvin Gaye and Diana Ross. Certainly no less famous artists became known by labels as Stax records in Memphis (Otis Redding, Isaac Hayes) and Philadelphia International in Philadelphia (the O’Jays, Harold Melvin & the Blue Notes). In their attempt to set them aside from the other labels, Stax called themselves the rural remedy for the sweet and polished sound of Motown (Borthwick & Moy, 2004; Gordon & Neville, 2007). Their sound was also referred to as “southern soul”. Philadelphia International tried to differentiate them in the same way by creating the so-called “Philly soul” (Seay, 2012). However, these differences in sound are not always as apparent as the labels might claim. It might even be the case that it was just a marketing strategy. This research aims to find empirical evidence that could prove or refute the claims made by these records labels by applying machine learning techniques. The research question reads as follows: to what extent is it possible to distinguish between American soul labels between 1959 and 1983 based on music content and using automatic classification techniques?

Three record labels will be considered; Tamla Motown, Stax Records and Philadelphia International Records. The next section will inquire more deeply into the specific characteristics of these three labels - music related and production related. Section 3 points out the applied method and approach. In section 4 the results are outlined, which will

(5)

5

then be given a musical interpretation in section 5. The final section contains the conclusion of this research.

2 Theoretic framework

2.1 Characteristics of record labels

Differences in the sound of record labels can have two origins; it can either be caused by attributes of the music or by the production process. Attributes of the music might be, for example, instrumentation and rhythm, whereas causes of production related differences could be the use of microphones and acoustics of the studio. It can even be said that the studio is an instrument itself. Manipulating the acoustics and tweaking the sounds with different mixing techniques enables producers to leave a signature on the sound of the music just as much as the musicians (Seay, 2012).

Three main dimensions of audio signals can be extracted from the music. The first dimension is timbre, which is the property of a sound wave that makes it possible to distinguish two voices or two instruments from each other. If production characteristics will become apparent, this will be through features in this domain. Secondly, melodic and harmonic features can be extracted. Melody is the property of music where pitch events are perceived separately and successively; harmony is the combination of multiple pitches in the same moment. The final dimension, which might be defined as some time related consistency in music, is rhythm (Scaringella, Zoia & Mlynek, 2006; Weihs et al., 2016).

Characteristics of the record labels can become apparent through the values of the features within these domains. However, these values do not yet have a musical meaning. To facilitate the interpretation of these quantitative results, information about the distinctive musical attributes of the music from each record label is needed.

2.1.1 Motown

The Detroit based label Tamla Motown is arguably the most famous of the four labels discussed in this research. In 1959 founder Berry Gordy transformed an old photo studio into the office of his new record label, called “Hitsville U.S.A”. The garage became known as studio A, controlled from a control room build in the kitchen (Justman et al., 2002). In 1971 Gordy moved the company to Los Angeles, leaving the famous studio and a group of Motown musicians behind. Since the high days of the typical Motown sound from then on were over, this paper considers only the period from the foundation of the company up to the move to L.A..

Before the company was relocated most of the records produced in that studio were supported by a fixed group of musicians called the Funk Brothers. In the documentary Standing in the shadows of Motown it is stated that “the Funk Brothers played on more number one hits than the Beach Boys, the Rolling Stones, Elvis and the Beatles combined” (Justman et al., 2002). Many statements about the characteristics of the Motown sound conclude that it was constituted by the musicians and their feel of the music (Justman et al., 2002; Schultz, 2009). Uriel Jones, drummer of the Funk Brothers, specifies that especially the bass lines of James Jamerson, who played bass with only one finger, and the drumming of Benny Benjamin formed the base of the typical sound of the Motown label (Schultz, 2009). Characteristics of Jamersons unique playing style are the warm and slightly overdriven sound and his use of dissonance and open strings. On the one hand he could make one note sound so strong and full as if it were multiple, but on the other hand he would play many notes without making it appear to a be a busy part (Jisi, 2009). Moreover, it is said that the arrangers only had general ideas about the chords and the feel and that the musicians

(6)

6

would complete the song in the studio (Fitzgerald, 2007; Schultz, 2009). Although Borthwick and Moy (2004) stress that the regularities in the sound should not be embellished, they have found several similarities in Motown recordings. Firstly, in the process of mixing treble frequencies from the tambourine, the xylophone and the hi-hat were brought forward. Also, many songs included a string section or a small orchestra and the usage of a walking bass style and blues chord progressions were highly uncommon. Furthermore, the rhythm guitar made use of a typical manner of staccato strumming, the so-called “chop” style, in order to put an emphasis on the backbeat (beat two and four in the bar) played by the snare drum. Finally, the sound of the drums was echoed in order to create a rich and vibrating environment (Borthwick & Moy, 2002; Fitzgerald, 2007). Fitzgerald (2007) adds that the call and response type of singing, the use of the pentatonic scale and a very extensive rhythm section were common features of Motown music.

Not only the musicians played an important role in the creation of the Motown sound, but also the songwriters and producers were of great significance. Especially in the early years of the company Berry Gordy and William “Smokey” Robinson formed the foundation of the writing and production team. While Gordy gradually turned only to his organizational role within the corporation, Robinson kept writing and producing songs until the end of the period studied in this analysis. When Brian Holland, Lamont Dozier and Eddie Holland one by one joined the label, the hit-making machine Holland-Dozier-Holland (H-D-H) was formed (Fitzgerald, 2007). In their songs riffs were often used as a tool to differentiate between the chorus and the verse. Additionally, choruses included short, catchy melodies and some memorable catchwords, while verses consisted of longer and more complex melodies. Love is a frequently recurring theme in the music of all the songwriters, though Robinson mostly expresses its joyful side and songs written by H-D-H are more negatively positioned. Also, H-D-H songs often have a fast tempo (around 160 beats per minute [bpm]), while Robinson seems to prefer more slowly paced rhythms (around 120 bpm). One final difference of writing style between these songwriters is the use of an AABA structure, which was regularly used by many Motown composers, including Robinson, but was mainly avoided by H-D-H (Fitzgerald, 2007).

2.1.2 Stax

In 1959 the siblings Jim Stewart and Estelle Axton decided to establish their own record label in an old movie theatre on McLemore Avenue in Memphis, called Stax Records. The music that came out of their studio, Soulsville U.S.A., was also referred to as southern soul. More than in the northern states of America, racial differences were an immense issue among the population and this was reflected in the raw and rural sound of the Stax music. Next to their studio, they built a record shop where they would let the costumers listen to new recordings (Gordon & Neville, 2007; Palmer, 1991) Although the company was active until 1976, the period studied in this paper stretches from its founding year to 1968, a year after the death of one of Stax’ biggest artists Otis Redding and the year that the distribution deal with Atlantic Records terminated (Borthwick & Moy, 2004; Bowman, 1995).

During this thriving period songs were principally written and produced by Steve Cropper, the duo Isaac Hayes and David Porter, Booker T Jones together with William Bell and by Otis Redding, many of which were also part of the label’s house band Booker T. and the MGs with the horn section named the Mar-Keys, who supported almost all of the Stax recordings (Gordon & Neville, 2007) These two consistent factors along with the use of only that one studio on McLemore, where the music was recorded live with all musicians present, were responsible for the development of southern soul (Bowman, 1995). In contrast to the Motown songs, a twelve-bar blues structure formed the basis for much of the music within this style, especially in the early years of the label (Borthwick & Moy, 2004; Bowman, 1995). A verse/chorus/bridge type of structure, arranged in an AABA manner, was also

(7)

7

frequently applied. The tempi are more moderately paced, generally belonging either to the category ballad, with tempi varying between 48 and 78 bpm, or mid-tempo, where the tempo lies between 102 and 132 bpm (Bowman, 1995). Before stereo recordings became possible in 1965, music was recorded through a single-track monophonic console. The same equipment such as microphones and amplifiers were used throughout the whole period discussed in this research. Lack of proper maintenance of this equipment could be the cause for the typical muddy sound. Additionally, Al Jackson’s drumming sound would intentionally situate the listener in a so-called ‘acoustically dead’ room without any reverberation (Borthwick & Moy, 2004; Bowman, 1995).

The instrumentation on Stax music is much sparser then on Motown songs, as they seemed to pursue the ‘less is more’ principle. The core consisted of a simple rhythm section with drums, bass, guitar and piano or organ in addition to a horn section consisting of three or four members. Multiple other instruments, such as strings and several percussion instruments, were added sporadically. The horn section would either provide harmonic support by playing swelling chords or replace the backing vocals in the call and response type of playing by answering to melodies of the solo vocalist (Borthwick & Moy, 2004; Bowman, 1995). While the snare drum played a distinctive delayed, lazy backbeat, the bass and bass drum would put an emphasis on beats one and three (Palmer, 1991; Schwartz, 1992). The use of cymbals was scarce and the hi-hat would be played in a closed style. The bass usually had the crucial task of playing riffs that formed the basis for the parts of every other instrument. By following the idea of ‘less is more’ and ‘playing the gaps’ a feeling of tension is build that leads to the creation of a notion called ‘groove’ that is present in most of this music (Borthwick & Moy, 2004; Bowman, 1995).

2.1.3 Philadelphia International

As stated by Borthwick and Moy (2004), Philadelphia International Records was arguably the last great soul label, yet extensive musicological research seems to lack. The company was founded in 1971 by Kenneth Gamble and Leon Huff, soon joined by partner Thom Bell, who were also the main songwriters and producers of the label. Their music, with its Motown-like pop influences and raw vocals similar to Stax music, would likewise be identified as Philadelphia soul or Philly soul. By 1983 the success days of the company were over and its distribution deal with the Columbia Broadcasting System had ended. This analysis will therefore stretch over the period between 1971 and 1983.

The band playing on their records was a group of almost forty musicians and got signed to the label in 1972 as MFSB, which stands for Mother Father Sister Brother. Only the horn section already consisted of ten artists and the sizeable rhythm section included no less than four guitarists, each with a specific job. While one guitar would answer the melody of the lead vocal, some other would complement the music with sounds such as the ‘wah-wah’ effect. Similar to the writing process of Motown, Gamble and Huff would come into the studio with general ideas for songs and the members of MFSB would “feel” what they were supposed to play. Typical for the sound of Philly soul was the use of “lush” strings, which can be heard on hit singles such as “Back Stabbers”1_{by the O’Jays and “The Love I}

Lost” by Harold Melvin and the Blue Notes (Jackson, 2004; Seay, 2012). These lush strings also became signifying for a new popular genre named disco. Although Gamble and Huff would not use the word ‘disco’, this new genre definitely had its influences on the music of Philadelphia International. It has even been stated that MFSB drummer Earl Young created the easily identifiable disco beat, with its four-on-the-floor bass drum and a quaver pattern on an open hi-hat connecting each beat (Jackson, 2004). Since this music was specially aimed towards dancing, with a tempo lying around 150 bpm, and with the emergence of

(8)

8

disco clubs, the production of music was no longer aimed towards radios or car speakers, but more to high fidelity home and club settings (Jackson, 2004; Seay, 2012).

Sigma Sound Studio was the home of all Philadelphia soul recordings. Owner of this studio and sound engineer Joseph Tarsia had a great influence on the sound of this music by tweaking the acoustics, both present in the room and manufactured by production techniques, and making smart use of the available resources (Seay, 2012). Tracks were always cut by first recording the rhythm section, then the vocals and concluding with the decorations of the horns and strings (Jackson, 2004). Since members of the rhythm section of MFSB preferred recording all together and sitting closely next to each other, isolation of instruments was hard to maintain, but Tarsia used the leakage between microphones as an advantage in creating the desired smooth and rich sound. Stressing the fact that Philadelphia International only came into existence when the here analysed time periods of the other two labels were already over, some production assets used on Philadelphia International songs were not yet available in those earlier years. These techniques, such as sixteen-track or even twenty four-track tape recorders, sound effects such as delay and reverb, noise reducers and console automation, which enables the engineer to save the arrangements of the faders, facilitated the recording of a much cleaner sound (Seay, 2012).

2.2 Genre classification

The classification of record labels within one genre is very little explored in the field of music information retrieval (MIR). However, genre classification is a problem that has been addressed many times and in many different ways and good results have been achieved (McKay & Fujinaga, 2004; Scaringella, Zoia & Mlynek, 2006; Sturm, 2013). Music libraries are becoming overwhelmingly large which makes it hard for people to find the music that they are interested in (Scaringella et al., 2006). Genre can serve as an important tool to organize music and help people discover what they are looking for (McKay & Fujinaga, 2004). Though, genre can be very subjective (Sturm, 2013). For example, it is related to marketing strategies and the time people live in. If a parallel could be drawn between sub-genres and record labels, discovering musical and production differences between different record labels could help making a step towards more objective genre classification.

In classification tasks three approaches can be handled, first of which is an expert system that follows a set of predefined rules. These rules should be able to characterise each genre, or in this case record label, very specifically, but given the complexity of such a task this procedure is not deemed suited for this research (Scaringella et al., 2006). Secondly the method of unsupervised learning, which groups the data in a number of non-predefined clusters based on similarities in the descriptions of the training examples, could be utilised. Since this technique could assemble the data in ways that do not correspond to the three investigated labels, as interesting as this might be, this approach will be excluded from this research. The final paradigm, supervised learning, is most suitable for this analysis as it attempts to find relations in the feature space of previously labelled data (McKay & Fujinaga, 2004). A more extensive description of the used algorithms can be found in the methodology section.

3 Method and approach

For this research a total of 1859 audio files are collected. 1450 of these songs belong to Motown and are extracted from the cd box ‘The complete Motown singles collection volumes 1-11’. The 244 files from Stax are derived from ‘The complete Stax/Volt singles: 1959 – 1968’, where Volt is one of Stax’ subsidiary labels, and the final 175 songs, released under the label of Philadelphia International, are obtained from ‘Philadelphia International

(9)

9

Records: The 40th Anniversary Box Set’. Volume 5 (1965) of the Motown collection could not be used for feature extraction due to technical difficulties and is therefore eliminated from this analysis. Of all 1859 examples, only the middle 40 seconds are withdrawn, leaving unforeseen long silences at the beginning and end of the file out of the research.

Feature extraction is executed in the MATLAB software environment using the MIRtoolbox, a commonly used toolbox in the field of Music Information Retrieval that enables the extraction of musical characteristics from audio files (Lartillot, Toiviainen & Eerola, 2008). 32 features are obtained within six domains. Besides the previously defined dimensions rhythm, timbre, harmony and melody, where harmony and melody here match tonal and pitch, the toolbox adds the domains dynamics and fluctuation, which are the alteration in loudness and the periodicity in rhythm respectively. Since not all feature outputs have the same dimensions, the mean and standard deviation of two-dimensional values are used. Also, the thirteen coefficients of the Mel-Frequency Cepstral Coefficients (MFCCs), the delta-MFCCs and the delta-delta-MFCCs are used as individual columns, which leads to 94 features in the final dataset. A list of the initial 32 features can be found in appendix B.

The selection of features as well as the classification of the labels is performed using the Python machine learning library scikit-learn (Pendrerosa at al., 2011). The classification is divided in four tasks:

1. Classifying Philadelphia International and Stax. 2. Classifying Motown and Stax.

3. Classifying Motown and Philadelphia International. 4. Classifying all three labels.

For each task, experiments are conducted using four different algorithms, all of which employ the method of supervised learning. A Support Vector Machine (SVM) is an algorithm that makes a clear separation in the feature space of the data, where each side of the separation gap represents a category, and classifies new data as the label it is mapped onto (Cortes & Vapnik, 1995). The other algorithms are ensemble methods that combine multiple models, in this case decision trees, in order to achieve a higher predictive value (Dietterich, 2000). Random Forest is an ensemble type classifier, which learns multiple Decision Tree models simultaneously and returns the most frequent label of all the predicted values of the models (Breiman, 2001). Gradient Boosting is a technique that iteratively tries to enhance a prior weak model by fitting a Decision Tree to its cost function and adding it to the model (Friedman, 2001). The final approach that will be used for the task of classification is Adaptive Boosting (AdaBoost). This algorithm assigns weights to each training example and then fits a series of weak Decision Tree learners where for each succeeding classifier the weights are adjusted in such a manner that the previously incorrect classified instances to become more emphasized (Freund & Schapire, 1995).

The conducted experiments for each algorithm make use of three different feature selection techniques in addition to the situation where no feature reduction method is employed. The most straightforward method is a Variance Threshold. This design, as stated in the scikit-learn documentation, removes all columns of which the variance is lower than some, by the user defined, boundary condition. The other two approaches are Recursive Feature Elimination (RFE) and Recursive Feature Elimination with Cross-Validation (RFECV). Both techniques first fit the data of all features using a specified algorithm, assigning each feature a weight that represents its importance for the task. In every step of the iterative process, the columns with the lowest importance are excluded from the data. While RFE selects a fixed amount of features, RFECV implements cross-validation in order to obtain an optimal number of columns (Pendrerosa et al., 2011). In all situations where RFE is applied 47 features will be selected. The three above-mentioned ensemble methods have the property of assigning importance weights to each feature. The three columns with the highest importance ranking will be retrieved for each experiment after the operation of feature reduction and will be used to determine which features are significant for each task.

(10)

10

Results are measured by the proportion correctly classified data points for each task. For evaluation 10-fold cross-validation is applied, returning the mean and the standard deviation. The baseline for each task is the situation where each description is classified as the label with the highest share within the group. Task 1 to 4 would by these means generate an accuracy of 0.58, 0.86, 0.89 and 0.78 respectively. The next section provides a full overview of the achieved results. In the discussion these results will be related back to the musical characteristics of each record label.

4 Results

Tables 1 to 4 show the accuracy results after tuning the parameters of the algorithms of classification tasks 1 to 4 respectively in terms of mean and standard deviation (std).

Table 1: Mean (std) results of 10-fold cross-validation for classifying Philadelphia International and Stax. Philadelphia vs. Stax None Variance Threshold RFE RFECV SVM 0.66 (0.12) 0.66 (0.12) 0.66 (0.12) 0.65 (0.12) Random Forest 0.94 (0.03) 0.94 (0.03) 0.96 (0.04) 0.96 (0.03) Gradient Boosting 0.94 (0.03) 0.96 (0.03) 0.96 (0.03) 0.97 (0.02) AdaBoost 0.95 (0.04) 0.95 (0.02) 0.97 (0.02) 0.97 (0.03) Table 2: Mean (std) results of 10-fold cross-validation for classifying Motown and Stax.

Motown vs. Stax None Variance Threshold RFE RFECV SVM 0.57 (0.19) (0.19) 0.57 (0.19) 0.57 (0.10) 0.48 Random Forest 0.92 (0.02) 0.93 (0.01) 0.93 (0.01) 0.93 (0.02) Gradient Boosting 0.95 (0.02) 0.96 (0.01) 0.96 (0.01) 0.96 (0.01) AdaBoost 0.96 (0.01) 0.96 (0.01) 0.97 (0.01) 0.97 (0.01)

Table 3: Mean (std) results of 10-fold cross-validation for classifying Motown & Philadelphia International. Motown vs. Philadelphia None Variance Threshold RFE RFECV SVM 0.55 (0.09) 0.55 (0.09) 0.55 (0.09) 0.68 (0.32) Random Forest 0.95 (0.01) 0.96 (0.01) 0.96 (0.01) 0.97 (0.01) Gradient Boosting 0.96 (0.01) 0.96 (0.02) 0.96 (0.01) 0.97 (0.01) AdaBoost 0.98 (0.01) 0.97 (0.01) 0.98 (0.01) 0.98 (0.01)

(11)

11

Table 4: Mean (std) results of 10-fold cross-validation for classifying Motown, Stax and Philadelphia International.

All labels None Variance

Threshold RFE RFECV SVM 0.49 (0.09) 0.49 (0.09) 0.49 (0.09) 0.58 (0.10) Random Forest 0.89 (0.02) 0.90 (0.01) 0.90 (0.01) 0.90 (0.02) Gradient Boosting 0.94 (0.02) 0.95 (0.02) 0.95 (0.02) 0.94 (0.02) AdaBoost 0.86 (0.03) 0.86 (0.02) 0.86 (0.02) 0.86 (0.03) In all four cases, the Support Vector Machines show the lowest performance, only achieving better results than the baseline in task 1 where Philadelphia International and Stax are the labels to be distinguished. The other three algorithms present significantly better results, with Gradient Boosting and AdaBoost returning the highest values. Employing an automatic feature selection method improved the results in most experiments, except for the situation in task 1 and 2 where a SVM was utilised. In every experiment where variance threshold was utilised thirty features are eliminated, which are all the delta-MFCCs and delta-delta-MFCCs, the mean and standard deviation of the attack times of the onsets and the standard deviations of the chromagram and the harmonic change detection function (HCDF). The overall highest values (mean of 0.98, standard deviation of 0.01) were obtained in task 3 where Motown and Philadelphia International were to be classified by implementing an AdaBoost classifier.

Highest performances for task 1 were achieved when using Gradient Boosting with RFECV reducing the feature space to 27 columns or AdaBoost with RFE, both realizing a mean of 0.97 and a standard deviation of 0.02 (table 1). The most important variables for this problem are:

 Mean of the onset curve

 Peak of the fluctuation spectrum

 Mean of the key clarity

 Standard deviation of the fluctuation spectrum

 Mean of the chromagram

 Standard deviation of the spectral flatness

Especially the mean of the onset curve ranked highest on every experiment.

Table 2 shows that the best results for categorizing Motown and Stax were realised when AdaBoost was handled with RFE and RFECV (mean is 0.97 and standard deviation is 0.01). For this situation RFECV selected 21 columns to contribute to the classification, four of which appeared to have the most predictive value:

 Mean of the spectral spread

 Mean of the spectral flatness

 Mel-Frequency Cepstral Coefficient 3

 Standard deviation of the spectral flux

The analysis of Motown and Philadelphia International produced the best results with a mean of 0.98 and a standard deviation of 0.01 in all experiments where AdaBoost is employed, except when a variance threshold is used, which can be viewed in table 3. Feature reduction seemingly did not enhance these results more than to the level of three decimals, even though RFECV reduced the feature space to 28 variables. Valuable features for this task seem to be:

 Mean of the onset curve

(12)

12

 Standard deviation of the fluctuation spectrum

 Standard deviation of the harmonic change detection function

Although the variance of the standard deviation of the HCDF is lower than 0.0001, causing it to be excluded by the variance threshold, results for this task are higher when this variable is included.

A mean of 0.95 and a standard deviation of 0.02 were the highest obtained results when categorizing all three labels, accomplished by handling a Gradient Boosting classifier with both the feature selection methods variance threshold and RFE, as stated in table 4. Of all four tasks these results are lowest and this is the only group of experiments where AdaBoost did not return the highest values. Columns that have a great predictive power are:

 Standard deviation of the fluctuation spectrum

 Mean of the onset curve

 Mean of the spectral flatness

 Standard deviation of the spectral flatness

 Standard deviation of the spectral flux

 Mean of the chromagram

All these variables have been stated before as being influential for classifying in the other tasks.

Every occasion where Philadelphia International is part of the labels to be categorized, the mean of the onset curve, the standard deviation of the fluctuation spectrum and the standard deviation of the spectral flatness stand out as to be significant. For Motown and Stax there are no such features that are prominent in all the situations where they are to be distinguished. As mentioned before, all features that have a great predictive power in task 4 also appear to be of value in other tasks, though not all the significant variables in task 1 to 3 occur in task 4. A more thorough description of all the discussed features will be given in the next section.

5 Discussion

The above-mentioned results indicate that there are most definitely characteristics of the three examined records labels that enable distinguishing them from each other. This section will go into more detail about these specific characteristics. There are eleven unique features, some of which appear as being important for multiple tasks and others are only significant in one specific task. The timbre domain is represented most in these predictors, namely five times, but this is in line with the proportion of timbre features in the complete set of features.

The only feature from the rhythm domain is the mean of the onset curve. An onset curve of an audio signal is a rhythmic feature that depicts energy peaks corresponding to the beginning of the notes in the music and can be used for tempo estimation. A higher mean of this curve would indicate a higher number of onsets in de audio fragment. Interestingly, this feature does not appear valuable in task 3, distinguishing Motown and Stax, although the tempi of these labels are quite far apart. As mentioned in the theoretic framework the tempo of a Stax song usually lays within either the category of 48 to 78 bpm, or between 102 and 132 bpm, while Motown tempi are either around 120 bpm or 160 bpm. Apparently some rhythmic attribute of Philadelphia soul causes this feature to jump out in each task where this label is considered. In fact, for Motown the values of this column mainly lay around 86 and for those of Stax are around 80, but the Philadelphia values are approximately 130. Possibly this is due to the disco influence on the label with its typical pumping four-on-the-floor bass drum pattern, instead of a more basic pattern of a bass drum playing on the first and third beat alternated with the snare drum playing the second and fourth beat. This disco beat supposedly was played for the first time on “The Love I Lost” by Harold Melvin and the Blue Notes in 1973 and can be heard after the introduction of 24 bars.

(13)

13

The domain timbre returned five significant features, two of which are related to the spectral flatness; an approach to measure the noise in the signal by considering the amount of peaks in the power spectrum (Dubnov, 2004). Values can range between 0 and 1, where 1 would indicate a high noise-like signal. The standard deviation and the mean of this feature thus express the change in noise and the average noise-level respectively. As stated in the result section, the standard deviation of the flatness appears in all experiments where Philadelphia International is part of the labels to be classified as being significant. The values of this column are higher for this label than those of Motown and Stax, indicating that there is more variation in the level of noise on Philly soul music. As stated in section 2.1.3, Philadelphia had the new production technique of noise reduction at its disposal. It seems probable that this new method not always reduced the same amount of noise in a recording, either due to it still being a new and rather experimental approach or to the fact that songs were recorded in several sessions, with the rhythm section, the vocals, the horns and the strings each doing a separate session. In contrast to Motown and Stax that always recorded in the same acoustic setting, the exact adjustments of the noise reducer might not have been identical in each session. The mean of this feature occurs in the task of categorizing Motown and Stax and in the final assignment where all labels are analysed. Surprisingly the values of this column for these two labels do not differ much, with the values of Motown being approximately 0.12 and those of Stax lying around 0.14. This difference, although small, might simply be a result of the different studios and equipment. As the theory states, the equipment at Stax lacked maintenance, which could account for these slightly higher values.

The timbre variable spectral spread returns the standard deviation of the spectrum at certain moments in time, describing the frequency range around its centroid. This feature can add a feeling of richness to a sound and high values are characteristic for noise-like signals (Weihs et al., 2016). The mean of this predictor seems relevant when categorizing Motown and Stax. Similar to the aforementioned spectral flatness, another noise level indicator, values in the data of Motown are slightly lower than those of Stax, but the Stax data is more widely distributed. As pointed out in the theoretic framework, in Stax audio the listener is placed further away from the instruments in an acoustically dead surrounding. Also, the inadequacy of maintenance of the studio equipment could further counteract a rich and full sound. This difference between these two labels, especially the absence and presence of reverb on the snare drum, can be heard on the song ‘My Girl’ played by the Temptations (Motown) and by Otis Redding (Stax).

Weihs et al. (2016) define spectral flux as “the amount of spectral change between consecutive signal frames” (p. 150). A high amount of this change in timbre would add a feeling of roughness to the music. The standard deviation of this predictor, representing the variation in timbre differences, appears valuable in the task of classifying Motown and Stax and in the final task where all three labels are considered. When looking at the data, values in this column belonging to Motown data are much lower than those of Stax and Philadelphia International. This would indicate that the level of change in timbre is much more constant in Motown music. Interestingly the mean of the spectral flux is generally higher for Motown and Philadelphia International than for Stax, while Stax actually praised themselves for their “rough” sound. A difference between intended musical roughness and unintended production roughness could support this. As mentioned before, Philadelphia International made use of new techniques, such as noise reducers, of which it was hard to replicate the exact settings in each recording session and that may not have been perfected at that time, which could account for these high mean spectral flux values. Moreover, it can be stated that Motown is the most consistent label in having a high amount of spectral change. A possible explanation might be the extensive instrumentation, which was not constantly present simultaneously, but instruments were added and removed throughout songs.

The fifth and final timbre feature, the Mel-Frequency Cepstral Coefficients (MFCCs), represents a characterization of the spectral shape by drawing on the nonlinear and logarithmic perception of pitch in the human ear (Hassan, Jamil, Rabbani & Rahman, 2004;

(14)

14

Lartillot et al., 2008). On the Mel scale frequencies below 1000 Hz are linearly spaced and those above 1000 Hz are logarithmically spaced, which is analogous to the human ear. These features are widely used for automatic speech recognition and have also proven to be very effective for MIR purposes (Hassan et al., 2004; Logan, 2000; Weihs et al., 2016). However, it is not known what specific aspect of timbre these coefficients analyse. In the second task, classifying Motown and Stax, the third MFCC component is returned as a significant feature and the data confirms this difference between these two labels, showing that the values for Motown are much lower than the Stax values. Still, an explanation based on the specific musical characteristics of these labels cannot be presented here.

Two of the eleven significant features concern the fluctuation spectrum, a variable that depicts the level of regularity at different frequency bands (Lartillot et al., 2008). The standard deviation of this spectrum is rated a high importance value in every task where Philadelphia International is considered and depicts the degree of rhythmic periodicity in the audio sample. High values would be yielded by extremely irregular songs and vice versa. The data shows that the Philadelphia values overall are lowest, suggesting that the music of this label has more rhythmic regularity than the other two labels. This could again be caused by the straightforward yet flexible disco beat with its four-on-the-floor bass drum and quaver hi-hat pattern. Also, Fitzgerald (2007) states that Motown songwriters Smokey Robinson and the group H-D-H included various riffs with complex rhythmically syncopated patterns in their music. Especially H-D-H incorporated many different riffs and rhythmic changes to emphasize the structure of a song. An example of such a rhythmic riff, played on guitar, can be heard in “You Keep Me Hangin’ On” by the Supremes. Moreover, most of the Stax music was groove-oriented, which entailed that the harmonic complexity was reduced to make room for more intricate rhythmic elements (Bowman, 1995). The peak of this spectrum displays the frequency band with the highest potential rhythmic periodicity and is a rough measure for the tempo of a song. This feature seems significant in task 1 where Philadelphia International is to be distinguished from Stax and examining the data shows that values for Stax are much lower. Based on the information in the theoretic framework, these results could be expected. It is stated that the tempi of Stax songs, with some exceptions, lay either between 48 and 78 bpm or 102 and 132 bpm, whereas the tempi of Philadelphia soul are generally around 150 bpm.

There are two important features from the tonal domain, which are the HCDF and the key clarity. The Harmonic Change Detection Function can identify alterations in the harmony, such as key shifts and chord progressions (Harte, Sandler & Gasser, 2006). The standard deviation of this feature, which appears valuable for distinguishing Motown and Philadelphia International, indicates the variety of harmonic change within an audio file. For example, songs with verses that contain a lot of different riffs and chords alternated with a simple two-chord chorus could return a high standard deviation of the HCDF. This is reflected in the writing style of the Motown songwriting team Holland-Dozier-Holland, who would use riffs to emphasise the difference between song sections by playing more riffs and complex melodies in the verse and a more straightforward, short and repetitive chorus statement accompanied by some memorable catchwords. “Can I get a Witness” by Marvin Gaye gives a good example of this writing style, where the verses are built on a twelve-bar blues structure, a repeating chordal pattern of generally twelve bars that consists of the first (tonic), fourth (subdominant) and fifth (dominant) chord of the key, while the choruses mainly consist of one chord and the repeated sentence ‘can I get a witness’.

The key clarity, the second variable from the tonal domain, calculates an estimation of the tonic and returns the strength of this corresponding key at certain moments in time (Lartillot et al., 2008). In task 1, where Philadelphia International and Stax are to be classified, the mean of this feature appears valuable, which is overall slightly higher for music of Philly soul then for Stax. This is a surprising result, since Philadelphia International made more use of an open hi-hat and cymbals, which could obstruct the clarity of the key. However, this label also employed more melodic instruments that again enhance the strength

(15)

15

of the key. Also, when listening to music from both record labels the noise-like sounds from the drums appear highly prominent on Stax recordings, while they seem to be overwhelmed by the harmonies of the backing vocals, strings and horns on music from Philadelphia International. This difference can be heard on the Philadelphia soul song “Dirty Ol’ Man” by the Three Degrees and the Stax recording “Soul Man” by Sam & Dave.

The final predictor, an (unwrapped) chromagram, originates from the domain pitch. This feature, widely used in chord recognition applications, displays the energy spread along different pitches (Lartillot et al., 2008). A fragment where all pitches are used to the exact same extent would return a value of 1. The mean of this spectrum, which is given a high importance rate in task 1 and 4, gives an indication of the variety of used notes, with higher values corresponding to a wider range of pitches. Philadelphia, Motown and Stax values for this column are around 0.15, 0.13 and 0.12 respectively, which corresponds to the amount of instruments used on the music of these labels. Philadelphia International is the label with the most extensive orchestration and has, because of the large string section with its ability to play high notes, the widest range in pitches. Strings are also employed on Motown recordings, but less frequently and are very scarce in southern soul. A horn and rhythm section are present at all three labels, but are again smallest at Stax. The full orchestration so typical for Philadelphia soul can be heard on “Back Stabbers” by the O’Jays.

6 Conclusion

This research aimed to find evidence that could prove or refute the claims of great soul record labels of having a sound that separates them from each other. The musical genre soul originates in 1960 from a combination of blues, gospel and jazz and became the voice of oppressed black America. Besides the many celebrated artists the sound of the genre was established through the vision of several influential record labels. In the 1960’s two major soul labels arose in the United States; one was a hit-making factory in the north called Motown, responsible for a polished and pop-influenced version of the genre and the other, Stax, created the raw and rural sound of southern soul. The third label considered in this paper with the name of Philadelphia International was founded in 1971 and took elements of both predecessors to produce its unique sound of Philadelphia soul.

Machine learning techniques were employed to find distinctive characteristics of each label based on music content. 94 features had been extracted from each of the 1859 samples in the dataset. The classification experiments were divided in four different tasks, the first of which aimed to distinguish Philadelphia International from Stax. The second task considered Motown and Stax en the third analysed Motown and Philadelphia International. The final task intended to classify all three labels. Four algorithms were tested (Support Vector Machine, Random Forest, Gradient Boosting and Adaptive Boosting) in four different automatic feature selection situations (no feature selection, variance threshold, recursive feature elimination and recursive feature elimination with cross-validation).

Best results (mean accuracy was 0.97 with a standard deviation of 0.02) for the first task have been achieved by applying Gradient Boosting with RFECV and AdaBoost in combination with RFE. The second problem realized a mean accuracy of 0.97 with a standard deviation of 0.01 when AdaBoost was handled with RFE and RFECV. The highest mean accuracy of task 3 was 0.98 with a standard deviation of 0.01 and was realized by employing AdaBoost either with no feature selection, RFE of RFECV. The final task yielded a best result of 0.95 (standard deviation: 0.02) when a Gradient Boosting classifier was used together with variance threshold or RFE.

Six features were significant for the task of classifying all three labels, three of which were also rated a high importance value in each task where Philadelphia International was considered, implying that these are characteristic for this label. The most significant

(16)

16

feature is the mean of the onset curve, which could be due to the driving four-on-the-floor bass drum pattern that was arguably first played by MFSB drummer Earl Young. A second valuable feature for this label, which depicts the change in the noise level of the song, is the standard deviation of the spectral flatness. Philadelphia International made use of production techniques such as noise reducers that weren’t available in the time of Motown and Stax. It might have been the case that the exact settings of these techniques could not be imitated in each recording session, resulting in a different level of noise reduction. The third and final distinguishing feature for the Philadelphia based label is the standard deviation of the fluctuation spectrum, which identified that this record label has the most rhythmic regularity, in contrast to Motown music that contains many rhythmically complex riffs.

There are several other important features when distinguishing all three record labels, the first of which is the mean of the spectral flatness. This feature was also returned in task 2 where Motown and Stax were considered, which could be a result of different recording equipment and lack of maintenance of that equipment at Stax. The second variable, the standard deviation of the spectral flux, was also rated valuable for classifying Motown and Stax. Motown appears to be the most consistent label in having a high amount of spectral change, which might be explained by the large orchestration at that label and the fact that these available instruments could be added and removed from different parts of the songs in any way the writer or producer pleased. The final highly relevant feature is the mean of the chromagram, a predictor that was also deemed important for categorizing Philadelphia International and Stax. The higher values obtained for this feature by the music of Philadelphia International could be explained by the considerable orchestration of this label, giving it the opportunity to use a wider range of pitches.

The findings of this research confirm that each of the three considered soul record labels has some musical characteristics that sets them apart from each other. Northern soul, southern soul and Philadelphia soul have clearly established three different sounds within one genre and they might even be labelled as individual sub-genres. These results support the idea of using record labels as a more objective means for automatic genre classification.

All three ensemble algorithms show great potential for this problem, with Gradient Boosting and Adaptive Boosting appearing to be the most suited. Further research could explore the capacity of other algorithms to enhance the obtained performance. Neural networks could be an interesting possibility, although more data would be needed to train such a model. However, within the field of Music Information Retrieval extensive datasets as are the standard in AI research are not always available. The here used dataset could be expanded, adding volume 5 of the Motown collection for completeness, but there is only a limited amount of music manufactured by each of the labels and this research shows that good results can already be obtained when employing a dataset of moderate size.

An extension of a human baseline to compare these computational results might be a good inclusion to add a new perspective to the results and to analyse the differences between the human ear and a computer. Additionally, more musicological into each of the records labels and the differences between them could be a basis for an a priori selection of features, which will be an interesting addition to further research. This could also be of help when analysing possible explanations for Motown being arguably the most famous label, based on the musical characteristics.

References

Borthwick, S., & Moy, R. (2004). Soul: from gospel to groove. In Popular music genres: An Introduction (pp. 5-22). Edinburgh: Edinburgh University Press.

(17)

17

Bowman, R. (1995). The Stax Sound: A Musicological Analysis. Popular Music, 14(3), 285-320.

Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.

Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.

Dietterich, T. G. (2000). Ensemble methods in machine learning. In International workshop on multiple classifier systems (pp. 1-15). Springer Berlin Heidelberg.

Dubnov, S. (2004). Generalization of spectral flatness measure for non-gaussian linear processes. IEEE Signal Processing Letters, 11(8), 698-701.

Fitzgerald, J. (2007). Black pop songwriting 1963-1966: an analysis of US top forty hits by Cooke, Mayfield, Stevenson, Robinson, and Holland-Dozier-Holland. Black Music Research Journal, 97-140.

Freund, Y., & Schapire, R. E. (1995). A desicion-theoretic generalization of on-line learning and an application to boosting. In European conference on computational learning theory (pp. 23-37). Springer Berlin Heidelberg.

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189-1232.

Gordon, R. & Neville, M. (2007). Respect Yourself: The Stax Records Story. United States: Tremolo Productions.

Harte, C., Sandler, M., & Gasser, M. (2006). Detecting harmonic change in musical audio. In Proceedings of the 1st ACM workshop on Audio and music computing multimedia (pp. 21-26). ACM.

Hasan, M. R., Jamil, M., Rabbani, M. G., & Rahman, M. S. (2004). Speaker identification using mel frequency cepstral coefficients. variations, 1(4).

Jackson, J. A. (2004). A house on fire: the rise and fall of Philadelphia soul. Oxford, United Kingdom: Oxford University Press.

Jisi, C. (2009). Secrets of the Motown Vault. Bass Player, 28-39.

Justman, P., Slutsky, A., & Passmann, S. (2002). Standing in the Shadows of Motown. United States: Artisan Entertainment.

Lartillot, O., Toiviainen, P., & Eerola, T. (2008). A matlab toolbox for music information retrieval. In Data analysis, machine learning and applications (pp. 261-268). Springer Berlin Heidelberg.

Logan, B. (2000). Mel Frequency Cepstral Coefficients for Music Modeling. In Proceedings of 1st International Society for Music Information Retrieval. Plymouth, MA.

Manzoor, S. (2005). Black and white. New Statesman, 134(4745), 52. Retrieved from http://go.galegroup.com.proxy.uba.uva.nl:2048/ps/i.do?p=LitRC&sw=w&u=amst&v=2.1&it =r&id=GALE%7CA133947387&asid=0113d077579950e208bdaf6e6cb534c9

(18)

18

Maultsby, P. K. (1983). Soul music: Its sociological and political significance in American popular culture. The Journal of Popular Culture, 17(2), 51-60.

McKay, C., & Fujinaga, I. (2004). Automatic Genre Classification Using Large High-Level Musical Feature Sets. In Proceedings of 5th_{International Society for Music Information} Retrieval (pp. 525-530). Barcelona (Spain).

Palmer, R. (1991). Danger: High voltage from Stax. Rolling Stone, (604), 109-111.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Vanderplas, J. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825-2830.

Scaringella, N., Zoia, G., & Mlynek, D. (2006). Automatic genre classification of music content: a survey. IEEE Signal Processing Magazine, 23(2), 133-141.

Schultz, D. (2009). Secrets Of the Motown Sound. Modern Drummer, 33(8), 84-88.

Schwartz, T. (1992). The Complete Stax/Volt Singles: 1959-1968. American Music, 10(3), 387-391.

Seay, T. (2012). Capturing That Philadelphia Sound: A Technical Exploration of Sigma Sound Studios. Journal on the Art of Record Production, 6.

Sturm, B. L. (2013). Classification accuracy is not enough. Journal of Intelligent Information Systems, 41(3), 371-406.

Weihs, C., Jannach, D., Vatolkin, I., & Rudolph, G. (2016). Music data analysis: Foundations and applications. CRC Press.

Discography

Various Artists (1991). The Complete Stax/Volt singles: 1959 – 1968 [CD]. Memphis, TN: Atlantic.

Various Artists (2005). The Complete Motown Singles/Vol 1: 1969-1961 [CD]. Detroit, MI: Motown Records.

Various Artists (2005). The Complete Motown Singles/Vol 2: 1962 [CD]. Detroit, MI: Motown Records.

(19)

19

Various Artists (2008). The Complete Motown Singles/Vol 11A: 1971 [CD]. Detroit, MI: Motown Records.

Various Artists (2009). The Complete Motown Singles/Vol 11B: 1971 [CD]. Detroit, MI: Motown Records.

Various Artists (2012). Philadelphia International Records: The 40th Anniversary Box Set [CD]. Philadelphia, PA: Philadelphia International Records.

Various Artists (2013). The Complete Motown Singles/Vol 12A: 1972 [CD]. Detroit, MI: Motown Records.

Various Artists (2013). The Complete Motown Singles/Vol 12B: 1972 [CD]. Detroit, MI: Motown Records.

Appendices

A Hardware and software specifications

All experiments are conducted using a MacBook Pro with a 2.5 GHz Intel Core i5 processor. Feature extractions in the MATLAB software environment are executed using MATLAB R2016b and the 1.6.3 version of the MIRtoolbox. Classification tasks were performed using the Python 2.7 version.

B Extracted features

The initial 32 extracted features using the MIRtoolbox (Lartillot, Toiviainen & Eerola, 2008).

Domain dynamics:

 Root-mean-square energy. Domain fluctuation:

 Summary of fluctuation spectrum;

 Peak of fluctuation summary;

 Centroid of fluctuation summary. Domain rhythm:

 Onset curve;

 Tempogram;

 Attack times of the onsets;

(20)

20 Domain timbre:  Spectrogram;  Spectral centroid;  Brightness;  Spectral spread;  Spectral skewness;  Spectral kurtosis;

 Roll-off using a threshold of 95%;

 Roll-off using a threshold of 85%;

 Spectral entropy;

 Spectral flatness;

 Roughness;

 Irregularity;

 Mel-Frequency Cepstral Coefficients;

 Delta-Mel-Frequency Cepstral Coefficients;

 Delta-delta-Mel-Frequency Ceptral Coefficients;

 Zero-crossing rate;

 Low energy rate;

 Spectral flux. Domain pitch:

 Chromagram;

 Peak of the chromagram;

 Centroid of the chromagram. Domain tonal:

 Key clarity;

 Mode;

Soul with a capital