Hindustani classical music revisited statistically: Does the order of Markov chain in the note dependence depend on the raga or the composition?

: Rendering musical notes randomly does not create music. To generate music, there has to be a pattern which makes the musical notes dependent. It is therefore of interest to know whether the probability of the next note depends on the current note only or whether it depends on the note(s) prior to the current note. In other words, it is important to explore the orderof Markov chain in the musical piece. In the context of Hindustani classical music, does this order depend on the raga or the composition? The present work addresses this fascinating question and attempts to answer it through Akaike’s information criterion (AIC). It appears, interestingly, that the order of Markov chain is dependent on the raga which has a well defined melodic structure with fixed notes and a set of rules characterizing a particular mood that is conveyed by performance. As long as these rules are maintained, as in a raga bandish, the order of the Markov chain is invariant over the raga compositions.


Introduction 1.Mathematics and music
Let us first take a look at the age old relationship between mathematics and music.Since the earliest civilizations, mathematics has existed in some form or other.Mathematics was employed by the Inca, the Egyptians, and the Babylonians, but it was not until the Greek Antiquity (600-300 BC) that it was studied for its own purpose.The study of mathematics is quite broad and has been for hundreds of years; many cultures and civilizations have explored, utilised, and researched the subject in various ways and manners.It is a subject that is always evolving, making it challenging to describe.Westerners' conception of mathematics in the twenty-first century is that it is the abstract science of amount, form, space, change and number.
Mathematicians use rigorous deduction to look for novel relationships and hypotheses.To solve problems, they apply logic, reasoning, and abstract thought.It is possible to study mathematics for its own sake or to use it to understand occurrences in other academic fields.For instance, physicists speak mathematically while describing the natural world.
Music, like mathematics, is an intrinsic part of human existence and has been a part of cultures throughout history.It is an artistic way of expressing emotions and ideas, and is frequently used to express and portray one's self and identity.Different forms of music are studied, performed, played and listened to.
A lovely subject that has been studied for hundreds of years is music theory.
Simply said, music theory is the study of the mechanics and elements of music.Analysis of any claim, notion, or statement made about or regarding music is one possible component.Musical notation (technically called score) and language are frequently studied by theorists of music.Music researchers look for trends and frameworks in compositional methods across or within genres, as well as across historical eras.
The fundamental generic definitions of mathematics and music suggest that they are two quite different fields of study.A discipline of science, often regarded as the queen of science, mathematics is characterised by order, countability, and calculability.Contrarily, music is seen as creative and expressive.Despite their apparent differences, these two fields of study are connected and have been so for over two thousand years.Music is inherently mathematical and many fundamental concepts in music theory are mathematical in nature.Like professionals in other fields, music theorists utilise mathematics to create, express and convey their ideas.
Numerous musical occurrences and ideas may be explained mathematically.Certain mathematical frequencies are used to describe how sound waves describe, how strings vibrate at particular frequencies etc. Instruments have some mathematical basis; for example, cellos have a certain form to mathematically resonate with their strings.Mathematics is also a crucial component of the contemporary technology that creates recordings for digital video discs (DVDs) and compact discs (CDs).These examples show how the connection between mathematics and music is intricate and evolving.
By looking at its various facets, this article tries to provide an overview of this complex link between mathematics and music.It is only fitting to start this report by briefly describing the historical connections between the study of music and mathematics.Inquiries into mathematics and physics have frequently been used throughout history to address questions and difficulties related to music theory.The second segment will go over some of the mathematics involved in music and sound.On the other hand, conceptions in music theory have frequently been directly impacted by mathematical ideas and language.There are several examples of composers who have induced mathematical concepts into their compositions.The third segment will go through Olivier Messiaen's "musical language's" mathematical strategies.
When explaining the illustrious relationship between mathematics and music throughout history, Pythagoras, Plato, and Aristotle were three exceptionally astute scholars and major influences.
Between around 600 BC and 300 BC, during the Classical Greek era, when Greece was made up of several city-states, Pythagoras was born.He was forced to flee his island home since it was ruled by a dictator.He established a mathematics religion (sometimes referred to as a cult) there.His religion's adherents, known as Pythagoreans, thought that mathematical constructions were magical.For further literature on the historical link between mathematics and music [1].

Hindustani classical music
Indian classical music has two streams: Hindustani (North Indian) and Carnatic (South Indian) and in either stream, the nucleus is the raga.A raga is a melodic structure with fixed notes and a set of rules characterizing a particular mood that is conveyed by performance [2].
The notes of every raga fall under one of the 12 chromatic tones of the justintonated scale, which is the basis for Hindustani music [3].The melodic notes of the Hindustani music scale are shadaj, rishabh, gandhar, madhyam, pancham, dhaivat, and nishad (or swaras, as they are called in other Indian languages).These notes are referred to as sargam collectively, which is comparable to the Solfè ge in the Western music system.The single syllable that is sung while singing is Sa, Re, Ga, Ma, Pa, Dha, and Ni; in this study, these notes are denoted by the symbols S, R, G, M, P, D, and N, respectively.Of these seven notes, Sa and Pa are always shudh (natural), while Re, Ga, Ma, Dha and Ni have two variants each, namely komal (flat) Re and Shudh Re abbreviated as r and R respectively, komal Ga and Shudh Ga abbreviated as g and G respectively, shudh Ma and teevra (sharp) Ma abbreviated as M and m respectively, komal Dha and shudh Dha abbreviated as d and D respectively and komal Ni and shudh Ni abbreviated as n and N respectively.If Sa is taken at natural C in western notation, the 12 notes of the octave in Indian notation are S r R g G M m P d D n N (capital letter is used for a natural note; small letter indicates a flat or a sharp note) which correspond to the notes C Db D Eb E F F# G Ab A Bb B respectively in western notation [4].
Hindustani music is tonal in nature and adheres to a hierarchy of notes, much like Western classical music [5].The tonic note is Sa or S. The scale only has seven natural notes, but each one of them can take two distinct forms, with the exception of S (the tonic) and P (the perfect fifth), which only have one form each.
In Hindustani music, each raga has a distinct ascending note pattern called "arohan," which progresses to the tonic of the following octave, and a comparable falling note pattern called "avarohan."According to Jairazbhoy [6], the arohan and avarohan are the most distinctive ascending and descending lines of a raga.In other words, the notes that should be played in a raga during rising and descending movements are indicated by the arohan and avarohan sequences.Arohan and avarohan singing and playing include a wealth of information that may be used to identify ragas.
As a result, arohan and avarohan sequences (i.e., the raga defining sequences) are created in the bandishes, studied and compared to the sequences noted in raga literature in order to confirm our representation [7].A raga bandish is a rhythmic song like raga composition that maintains the raga rules correctly.

Why are patterns and structures important in early mathematics?
"There is geometry in the humming of the strings, there is music in the spacing of the spheres."-Pythagoras.
Mathematical patterns follow a predictable norm that enables us to forecast what will happen next, while they frequently also have a pleasing aesthetic appeal.According to mathematicians, mathematics is the study of patterns, including patterns and structure in geometry as well as patterns and structure in numbers.
Time signatures, overtones, tone, pitch, scales, intervals, symbols, and harmonies too have interesting patterns.Mathematics is tied to the notations used by composers and the sounds produced by performers.Think about the connections between mathematics and music the next time you hear or play classical, rock, folk, religious, ceremonial, jazz, opera, pop, or current music.Consider how mathematics is utilised to produce the music you like.Patterns are where mathematics and music most resemble each other.Music, for instance, frequently repeats its choruses and verses, while mathematics employs patterns to explain the unknowable.
Different mathematical phenomena can be used in music.These include trigonometry, differential calculus, signal processing and even geometry.In fact, studies have found that music tends to be more well-liked when it exhibits some sort of mathematical structure.
According to research on music and music therapy, mathematics and music are linked in the brain from an early age [8].Mathematical concepts like spatial characteristics, sequencing, counting, patterning, and one-to-one correspondence are fundamental to musical aspects including a constant beat, rhythm, melody and space.The extremely primitive areas of the brain also appear to be connected to music.Physiological responses to music are inevitable in humans [9,10].This suggests that even the smallest kids may be able to respond naturally to music and the mathematical ideas it provides.
Recent studies in neuro-musicology have shown that human attention behaviours are influenced by a consistent rhythm.The premotor cortex of the brain, which is also connected to attention, is where we normally process steady beat.120 babies between the ages of 5 and 24 months were found by to be more attentive to rhythmic stimuli than speech-only stimuli.The findings of this study suggest that youngsters may be more attentive to instructions while listening to consistent beats as opposed to vocal instructions.As a result, it is possible that listening to a constant beat pattern while engaging in mathematics-related tasks with young children in an early childhood school can help them pay attention and become more engaged [11].

Computation of the transition probability matrix (TPM)
Probability theory does not directly explain the decision process of the artist as music is always planned and not random.However, from the listener's or the analyst's perspective (the latter being our case), this deterministic response may be realized as the outcome of a stochastic process [2].Further, as music must have patterns, the random variables are dependent.
We gave each raga a matrix representation that quantified the transition between the swaras (notes).As a result, the representations are referred to as transition probability matrices (TPM).Next, we carefully detected musical passages where the performers utilised swaras other than the middle octave by transcribing the performances of two musicians' rendering the same raga.The phrases were then moved from the lower (or higher) octave to the middle (or higher) octave.We were able to record information about transitions from one swara to another, which was our primary interest, despite losing information about the octave where the musicians preferred to sing or play a particular combination of swaras, thanks to this operation.In addition, our matrix representation would have been quite sparse and the transition information between the swaras would have been less helpful if we had used performances from only two performers that covered all the octaves.
Even though very few ragas employ all 7 of the swaras in Hindustani music, we took them all into consideration while creating the matrix.However, this would record two different types of information: a) a quick glance at the matrix will show which swaras are used in any given raga, and b) because some ragas have unidirectional note transitions the matrix will represent the entire raga rather than a specific direction of movement.
We calculated how frequently any one specific swara was followed by another swara by calculating the precise number of transitions from one swara to the other swaras after we had each individually constructed a corpus of swaras transcribed from each raga.In order to calculate the frequency of transitions from each swara to each of the 7 swaras, including transitions to the same swara, we repeated this technique for all 7 swaras.The total of all transitions emanating from any one particular swara was divided by that number.This represented the likelihood of a swara appearing in relation to the swara that initiated the transition.

Validation of the TPM representation using raga classification
Suppose we collected 10 swara-sequences from each raga to assess the reliability of TPMs as a representation for Hindustani raga music.We estimated the chance that each given swara-sequence belonging to one of those 10 ragas by applying the method for computing swara-sequence score, which is discussed in the methods section.The parent raga that could produce a sequence like the test swara-sequence was thought to be the TPM that produced the greatest score.Thus, all 100 swara-sequences were divided into 10 ragas based on the calculated score.Due to the fact that we obtained the swara-sequences from various interpretations of the ragas that were taken into consideration for this study, we examined whether the raga identification based on the TPM score matched the raga name.It is interesting to see that 100% of the swara sequences were accurately categorised into their parent ragas.The excellent classification accuracy achieved with random swara-sequences demonstrates that TPMs may be utilised to identify ragas [12].

Relationship between ragas
We determined the Euclidean distance between the ragas using the TPMs in order to comprehend the raga relationship and assess the effectiveness of TPM as a raga representation.We then used the traditional multidimensional scaling approach to transfer the connections between the ragas to a two-dimensional spatial representation based on those Euclidean distances as a metric.In addition to measuring distance, the use of swaras in a raga was examined in order to better understand the link between the ragas.We summed up all instances of each swara throughout the recital and determined the proportion of each raga's appearances that each swara is made up.We next determined the correlation coefficients between ragas using the percentages of occurrence of swaras for all ragas as a gauge of similarity.We reasoned that more related ragas would use the same notes more frequently, leading to higher frequency for that note and ultimately higher correlation coefficients between ragas.This might offer crucial details regarding the ragas' tonal hierarchies, as demonstrated by Castellano et al. [5].
MATLAB was used for all calculations pertaining to the development of TPMs, including the verification of TPMs as a representation for Hindustani ragas and the analysis of raga-to-raga correlations.

Mathematical formulation of the problem
We employed a Markov chain as our model to capture the statistical regularities between swaras in Kafi raga, which is the parent raga of the Kafi thata (thata is a raga group according to scale, ragas of Kafi thata have komal Ga and komal Ni and the rest five notes all shudh), due to the dynamic character of Hindustani music and the linkages between swaras in a swara sequence.Markov models have been widely used to speech and music including both Western and Indian classical music.

Computation of unconditional and conditional probability
The unconditional probability of a note is computed as the relative frequency, that is, by taking the ratio of number of times this particular note occurs in the musical piece to the total number of notes in the musical piece.
The conditional probability of the next note Y given the current note X = P(Y/X) = (number of times X is followed by Y)/(number of times X occurs in the musical piece).However, if the last note in the musical node is X, we have to subtract one from the denominator because there is no information of the next note transition for the last note.It is also important to note that a probability has to be expressed as it is and not by simplifying the fraction.The reason is that such simplification will change the musical meaning of the note probability (for example, if the probability of the tonic Sa is, let us say, 30/100 we must not simplify it to 3/10.The former ratio implies that there are 100 notes in the musical piece out of which 30 occurrences are of the tonic Sa.The latter ratio implies that there are just 10 notes in the musical piece with 3 tonic occurrences!Thus, although both the ratios are mathematically equal, they bear different meanings musically).
In case of Markov chain of order 1st we have taken the number of model parameters = k = 3 assuming the model Xn+1 = β0 + β1Xn + ε (1) The three parameters are β0, β1 and the error variance.In case of Markov chain of order 2nd we have taken k = 4 assuming the model Xn+1 = β0 + β1Xn + β2Xn−1 + ε (2) and the four parameters are β0, β1, β2 and the error variance.
(3) These probabilities may be written in the matrix form This is called the transition probability matrix or matrix of transition probabilities (TPM) of the Markov chain.P is a stochastic matrix i.e., a square matrix with nonnegative elements and unit row sums.

Determination of the order of a Markov chain by AIC
A procedure for the determination of the order of a Markov chain by Akaike's information criterion (AIC) has been developed by Tong [13] .The AIC is defined as AIC = (−2)ln(Maximum-likelihood) + 2 (number of independent parameters in the model) (5) AIC = (−2) ln (Maximum-likelihood) + 2k (6) where, k = number of independent parameters in the model.This statistic is introduced as a measure of deviation of the fitting model from the true structure.
Given several models, the procedure envisages adoption of the model that minimizes the AIC and is called minimum AIC estimation (MAICE).(It is argued that the MAICE procedure represents an attempt to strike a balance between overfitting which needs more parameters, and underfitting which incurs an increased residual variance).
Denote the transition probability for a r-order Markov by pij...ki, i = 1, 2, ..., s, s being the (finite) number of states of the chain.
The hypothesis Hk states that the chain is k-dependent while the hypothesis Hr states that the chain is r-dependent for k < r.Denote by λk,r the ratio of the maximumlikelihood given Hk (that the chain is of order k) to that given Hr (that the chain is of order r); then we get  ,r =  k,k+1 + ⋯ +  r−1,r (9) and so     = −2log ,+1 − ⋯ − 2 log  −1, ,  <  (10) Assume the variables −2log λr−1, r (r = 0, 1, ...) to be asymptotically independent, given Hk; then it follows that kAr has χ2-distribution with d.f. +1 −  +1 , k ≥ 0 (11) and  +1 , k = −1 (12) where   =   −  −1 , r ≥ 1 (13) Now the question of choice of an appropriate loss function arises once this identification procedure is considered as a decision procedure.The loss functions considered in classical theory of hypothesis testing are defined by the probabilities of accepting the incorrect hypothesis or rejecting the correct hypothesis.
Tong proposes the choice of the loss function, based on the AIC approach as () =    − 2(∇ +1 − ∇ +1 ) (14) where m is the highest order model to be considered and k is the order of the fitting model.The MAICE of the best approximating order of the Markov chain is that value of k which gives the minimum of R(k) overall orders considered.
Note that R(m) = 0. (15) Gabriel and Neumann described the occurrences and non-occurrences of rainfall (of Tel Aviv) by a two-state Markov-chain.A dry date is denoted by state 0 and a wet date by state 1.For further literature on stochastic process, refer to Medhi [14], Ross [15], Parzen [16], and Cinlar [17].
Our problem is to determine whether the order of the Markov chain in a raga depends on the inherent melodic structure of the concerned raga or does it vary over compositions in the same raga composed by different composers/musicians?We address this question using AIC taking different compositions (raga bandishes) in raga Kafi.More precisely, we shall be considering two famous raga Kafi bandishes' notations for comparing the first and second order of Markov-chain using Akaike's information criterion (AIC).

Does the order of Markov chain depend on the raga or its composition?
Indian and corresponding western notes are given below:
Table 2. TPM of the first Kafi bandish assuming Markov chain of second order.Table 3 gives the TPM of the second Kafi bandish assuming Markov chain of first order.

Discussion on the experimental results
For the first raga Kafi bandish, we observe that: AIC for 1st order of Markov-chain > AIC for 2nd order of Markov-chain as 100.98 > 65.63.
Therefore, the second order Markov chain corresponds to a better model that represents the raga structure.To put it simply, the probability of the next note depends not only on the current note but the previous note also.
For the second raga Kafi bandish, we again observe interestingly that: AIC for 1st order of Markov-chain > AIC for 2nd order of Markov-chain as 85.26 > 40.14.
Therefore, once again the second order Markov chain corresponds to a better model that represents the raga structure compared to the Markov chain of first order.Given that increasing the order of Markov chain may not lead to a better composition as warned very clearly by Nierhaus [18], we shall take the second order Markov chain to be a signature for representing the note dependence in raga Kafi.

Concluding remarks
It appears, interestingly, that the order of Markov chain is dependent on the raga which has a well defined melodic structure with fixed notes and a set of rules characterizing a particular mood that is conveyed by performance.As long as these rules are maintained, as in a raga bandish, the order of the Markov chain seems to be invariant over the raga compositions.However, we propose to extend this study to other ragas and compositions thereof for substantiation.

Table 1 .
TPM of the first Kafi bandish assuming Markov chain of first order.

Table 3 .
TPM of the second Kafi bandish assuming Markov chain of first order.

Table 4
gives the TPM of the second Kafi bandish assuming Markov chain of second order.

Table 4 .
TPM of the second Kafi bandish assuming Markov chain of second order.
our analysis.Source of the musical data: Raag Kafi Parichay and Bandish Notation [19,20].Conceptualization, SC; methodology, SC and AH; software, AH and PS; validation, SC, AH and PS; formal analysis, AH and PS; investigation, AH and PS; resources, AH and PS; data curation, AH and PS; writing-original draft preparation, SC, AH and PS; writing-review and editing, SC; visualization, SC and AH; supervision, SC; project administration, SC; funding acquisition, SC.All authors have read and agreed to the published version of the manuscript.