-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
454 lines (396 loc) · 22.3 KB
/
Copy pathindex.html
File metadata and controls
454 lines (396 loc) · 22.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
<html>
<head>
<link href="https://fonts.googleapis.com/css?family=Fjalla+One|Open+Sans+Condensed:300|Ubuntu|Yanone+Kaffeesatz" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Fjalla+One" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Fjalla+One|Open+Sans+Condensed:300" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Fjalla+One|Open+Sans+Condensed:300|Ubuntu" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Cuprum|Fjalla+One|Open+Sans+Condensed:300|Ubuntu|Yanone+Kaffeesatz" rel="stylesheet">
<style type="text/css">
body{
background-color: #e1e1ea;
background-image: url("Background3.jpg");
background-position: top;
background-repeat: no-repeat;
background-size:1350px 400px;}
p{display: inline-block;}
img{display: block;}
.container{width: 90%;position absolute;margin: auto;}
.title{position: relative;width: 90%;margin: auto;text-align: center;font-weight: bold;font-size: 90px;padding: 1%;font-family: 'Yanone Kaffeesatz', sans-serif; color: yellow;}
.section{position: relative;width: 90%;margin: auto;padding: 2%;}
.subsection{position: relative; width: 98%;text-align: justify;padding: 10px;}
.heading{position: relative; width: 98%;text-align: left;font-size: 30px;font-family: 'Ubuntu', sans-serif;}
.text{width: 95%;font-size: 20px;text-align: justify;padding: 10px 0px 10px 0px;font-family: 'Open Sans Condensed', sans-serif;}
.authors{position: relative;width: 80%;margin-left: 3cm;margin-top: 3.6cm;text-align: center;padding: 2%;font-size: 20px; font-family: 'Cuprum', sans-serif; color: yellow;}
.image{width: 95%;font-size: 20px; text-align: left;}
.image2{width: 95%;font-size: 20px; text-align: right;}
table, td, th {
border: 1px solid #ddd;
text-align: left;
}
table {
border-collapse: collapse;
width: 100%;
}
th, td {
padding: 15px;
</style>
</head>
<body>
<div class="container">
<div class="title">COMMERCIAL DETECTION IN AUDIO</div>
<div class="authors">
<!-- Start edit here -->
<p>Abhishek Gupta 150108048 EEE |</p>
<p>Suyash Pati Tripathi 150108038 EEE |</p>
<p>Ankur Deka 150102006 ECE</p>
<p>Parth Pawar 150108022 EEE |</p>
<p>Nitesh Khichi 150102042 ECE</p>
<!-- Stop edit here -->
</div>
<div class="section">
<div class="heading">Abstract</div>
<div class="text">
<!-- Start edit here -->
In this work, we have developed a system that is able to detect and remove commercials from an input
sound stream (radio broadcasts). Various features like spectral flux, zero crossings, spectral centroid,
signal energy etc., are extracted from the audio. Then these features are fed to Kalman filter, which sets
the value of state variable (predicted-state). Using a decision boundary with the state variable, each frame
is classified as either advertisement or song.
<!-- Stop edit here -->
</div>
</div>
<div class="section">
<div class="heading">1. Introduction</div>
<div class="text">
<!-- Start edit here -->
Advertisements have become ubiquitous and form a major
portion of radio broadcasts in modern times. Many listeners
are disinterested in these commercials as they tune in to listen
to radio programmes. The primary revenue stream for social
and entertainment media is advertisement and entities like
YouTube and Google depend on them for revenue.
<br>However,
users of these social media do not like the commercial
breaks; not only is this distracting and annoying for listeners, it
also has adverse effects on consumer behavior .
<br>In this work
a radio stream monitoring system is designed and
implemented to detect these advertisements.
<!-- Stop edit here -->
</div>
<div class="subsection">
<div class="heading">1.1 Introduction to Problem</div>
<div class="text">
<!-- Start edit here -->
This project aims at identifying and removing commercials segments from
audio signals acquired from radio feeds. The audio feed is first segmented
into a number of consecutive audio clips.
<br>The task is to identify specific
features that differentiate a commercial clip from a non-commercial one.
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">1.2 Figure</div>
<div class="image">
<!-- Start edit here -->
<br>Block diagram of the project.
<img src="example.jpg" alt="This text displays when the image is umavailable" width="800px" height=""/>
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">1.3 Literature Review</div>
<div class="text">
<!-- Start edit here -->
<ol>
<li><a href="https://www.ee.columbia.edu/~dpwe/papers/ScheiS97-mussp.pdf">
CONSTRUCTION AND EVALUATION OF A ROBUST MULTIFEATURE SPEECH/MUSIC DISCRIMINATOR by <i>Eric Scheirer
and Malcolm Slaneyy, MIT Media Laboratory, Cambridge, USA</i></a>
<br>This paper deals with the construction of a real-time computer system capable of distinguishing
speech signals from music signals over a wide range of digital audio input.</li>
<li><a href="https://www.ee.iitb.ac.in/~esgroup/es_mtech04_sem/es_sem04_paper_04307909.pdf">
AUDIO SIGNAL CLASSIFICATION by <i>Hariharan Subramanian,M.Tech. Credit Seminar Report IITB</i></a>
<br>This paper explains in detail various features and their use in Audio Signal Processing.</li>
<li><a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.217.3866&rep=rep1&type=pdf">
DETECTING ADVERTISEMENT IN RADIO USING MACHINE LEARNING by <i>Robin M¨uller-Cajar,University of Canterbury, Christchurch,
New Zealand</i></a>
<br>This paper explains the use of Machine Learning and related algorithms in Audio Signal Processing. It also explains in details
about various features and their use.</li>
<li><a href="https://courses.engr.illinois.edu/ece420/sp2017/UnderstandingKalmanFilter.pdf">
Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation by <i>Ramsey Faragher, IEEE Signal
Processing Magazine, September 2012</i></a>
<br>This article discusses in details about the basics of Kalman Filter.</li>
</ol>
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">1.4 Proposed Approach</div>
<div class="text">
<!-- Start edit here -->
We focused on identifying key features in the audio stream that are only present in advertising.
These features need to be consistent across most, if not all, advertisements, while
also being simple to calculate.
<br>Thus our research is split into several key steps:
<ul>
<li>Research and discover features that differ between advertising and music</li>
<li> Implement algorithms that can extract attributes corresponding to these features</li>
<li>Use Kalman Filter algorithms to create a function with these attributes as
inputs, and the class (advertising or music) as output</li>
</ul>
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">1.5 Report Organization</div>
<div class="text">
<!-- Start edit here -->
Our report starts with the understanding and analysis of advertisements.
<br>This results in the discovery of key features that can be used to distinguish between advertising
and music.The condensed block diagram of our proposed design is shown section 1.2 followed by various research papers and
related resources used in this project in section 1.3.
<br>The detailed approach is given in section 2.
<br>We implement algorithms to extract
these features before describing the design and implementation of Kalman Filter
algorithms in section 3.
<br>The results of testing the Kalman Filter algorithms are described in 3.1.
along with the quality of the extracted attributes.
Our discussion section 3.2 then attempts to explain each classifier and its performance.
<br>Finally we discuss our conclusions, summary and future work in section 4.
<!-- Stop edit here -->
</div>
</div>
</div>
<div class="section">
<div class="heading">2. Proposed Approach</div>
<div class="subsection">
<div class="heading">2.1 Analysing Advertising</div>
<div class="text">
<!-- Start edit here -->
Our study focused on commercial radio stations, through which we were able to find several
attributes that differed from music.<br> We recorded several hours of radio from 93.5 Big FM,
Vividbharti, RED FM as test data. These recordings were taken at different times of day
between 8am and 12pm in the last week of Octuber. <br>The advertisements that were captured in this
manner were then analysed and compared to music in the same segments.
<br>This gave us the following key attributes:
<ul>
<li><b>Speech:</b> A significant portion of advertisements contained only speech, while
some contained either music or other sound effects in the background.</li>
<br><li><b>Rhythm of Speech:</b> Compared to music and the DJ, advertisements sometimes appeared
rushed, as traders attempted to squeeze as much information into as little time as possible.</li>
<br><li><b>Frequency Distribution:</b> Advertisements contain fewer frequencies,
but those frequencies have been compressed to make the audio appear
louder. The range of frequencies tends to be smaller than for music, when the
advertisement consists of pure speech. For all advertisements the frequency
balance is skewed more towards lower frequencies.
<br><li><b>Volume and Volume Distribution:</b> The silent gaps between words mean that the
average volume of speech is lower. Also the change in volume over time is
higher for advertisements for the same reason</li>
<br><li><b>Length of Segment:</b>An advertisement is never longer than a minute, while music
is significantly longer.</li>
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">2.2 Audio Analyser</div>
<div class="text">
<ul>
<li><b>Spectral Analysis:</b> Music usually uses more frequencies than speech.
These frequencies tend to change rapidly as new notes are played. This contrasts with
speech where the speaker tends to remain within a thin frequency spectrum, with most
frequencies remaining empty. Thus if we sum how much each frequency changes
from one sample period to the next, we can gather a value for how much the audio has changed
in that period. This value will be higher for music or noise than for speech.
<br>The following classifiers were used:
<ol>
<li><b>Spectral flux analysis</b></li>
<div class="image">
<img src="Flux.png" alt="This text displays when the image is umavailable" width="229px" height="80"/>
</div>
<br><li><b>Spectral Centroid</b></li>
<div class="image">
<img src="Centroid.jpg" alt="This text displays when the image is umavailable" width="229px" height="80"/>
</div>
<br><li><b>Spectral Roll-Off Frequency</b></li>
<div class="image">
<img src="RollOff.png" alt="This text displays when the image is umavailable" width="229px" height="80"/>
</div>
<br><li><b>Spectral Flatness/Spread</b></li>
<div class="image">
<img src="Flatness.png" alt="This text displays when the image is umavailable" width="229px" height="80"/>
</div>
</ol>
<br><li><b>Extreme Frequency Values:</b>This value measures the volume distribution of the audio stream.
Advertising contains some very strong frequencies because it tends to heavily emphasise
the speech frequencies, something that music usually does not. By summing the number of frequencies
above and below a certain threshold we obtain a good estimation.</li>
<br><li><b>Energy Balance:</b>The energy balance compares the sum of the frequencies above 8000Hz with the
total sum of all frequency values. This value is significant because speech contains few frequencies
above 8000Hz while instruments contain significant amounts of frequencies in this range. This does
not always hold as certain music genres such as rap often contain only speech and drums, and therefore
very few high frequencies for the general volume distribution.</li>
<br><li><b>Zero crossing:</b>Music is more continous and smooth as compared to Audio advertisement. Speech contains pauses
unlike music. </li>
<div class="image">
<img src="zcr.jpg" alt="This text displays when the image is umavailable" width="229px" height="80" margin-left:1cm/>
</div>
<br><li><b>Volume:</b>The average volume for speech tends to be lower than for music, as speech contains
pauses, which music does not. Volume patterns tend to be more constant for music than for speech,
as the latter contains many more peaks and troughs in each sample period.</li>
<div class="image">
<img src="Volume.png" alt="This text displays when the image is umavailable" width="229px" height="80"/>
</div>
</ul>
</div>
<div class="subsection">
<div class="heading">2.3 Kalman Filter Algorithms - Design and Implementation</div>
<div class="text">
Kalman filter is an optimal estimation algorithm Kalman filter is used to estimate a system state when it cannot be measured directly
Also it is used to estimate a state of the system by combining measurements from different sources that may be subject to noise.
<br>The best thing about this filter is it’s simple mathematical structure which makes it feasible to be applied in real
time systems.
<br>There are essentially 2 steps in a Kalman filter –
<ol>
<li><b>Motion/Time Update step</b></li>
<li><b>Measurement Update Step</b></li>
</ol>
<div class="image">
<img src="Kalman1.jpg" alt="This text displays when the image is umavailable" />
</div>
<br><b>Use of Kalman Filter in Commercial Detection:</b>
We make use of a simple 1D Kalman filter which is equivalent to <i>Bayesian Filter.</i>
Our state is a 1D variable X with associated uncertainly (variance) P. An audio file is read
in clips of 3 seconds and with every clip, the state is updated (through one motion update and
one measurement update). If the state comes as positive then the clip is considered as commercial
and if it comes as negative it is considered music.
<br>In every clip:
<ol>
<li>Features are extracted, and these features are used as sensor readings.</li>
<li><b>Measurement Update:</b> The sensor readings lead to a measurement in state. X and P
are updated.</li>
<li><b>Motion Update:</b> The state is moved closer to origin and the uncertainly (P) is increased.</li>
</ol>
<br>Here is the justification for the approach:
<ol>
<li>Since previous state has certain amount of say in the next state, the erroneous detection of commercials are
filtered. The Kalman Filter in some sense is preventing sudden changes in state. E.g. It is unlikely that
a single clip of 3 seconds is commercial while all the neighbouring clips to the left and right are music.</li>
<br>
<li>If an advertisement is running for a long time than it is likely that it will end soon. Similar is the case with
music. The motion update step moves the state towards origin (i.e., commercial towards music and music towards
commercial).</li>
</div>
</div>
</div>
<div class="section">
<div class="heading">3. Experiments & Results</div>
<div class="subsection">
<div class="heading">3.1 Dataset Description</div>
<div class="text">
<!-- Start edit here -->
The datasheet are various audio sigmals in different radio channels. We recorded them using the <i>RadioSure</i> app.
These recordings are mailnly from 93.7MHz BigFm, 106.4MHz Vividbharti and some local channels.
These signals were taken at regular interval between 09:00 to 22:00 for a week.
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">3.2 Observation</div>
<div class="text">
<!-- Start edit here -->
The various graphs describing the feature analysis based on different Audio Analysers are shown below.
</div>
<div class="image">
<br>Zero Crossing
<img src="Zerocrossing.png" alt="This text displays when the image is umavailable" width="700px" height=300"/>
</div>
<div class="image">
<br>Spectral Spread
<img src="SpectralSpread.png" alt="This text displays when the image is umavailable" width="700px" height="300" />
</div>
<div class="image">
<br>Spectral Roll-Off Frequency
<img src="SpectralRollOff.png" alt="This text displays when the image is umavailable" width="700px" height="300" />
</div>
<div class="image">
<br>High Frequency Energy
<img src="HighFreqEnergy.png" alt="This text displays when the image is umavailable" width="700px" height="300"/>
</div>
<div class="image">
<br>Spectral Flux
<img src="SpectralFlux.png" alt="This text displays when the image is umavailable" width="700px" height="300"/>
</div>
<div class="image">
<br>Spectral Centroid
<img src="SpectralCentroid.png" alt="This text displays when the image is umavailable" width="700px" height="300"/>
</div>
<!-- Stop edit here -->
</div>
<div class="heading">3.3 Results</div>
<p>The results obtained on applying Kalman filter on the extracted features are shown below:
<br>
<table>
<tr><th>Feature</th><th>Kalman Filter Output</th><th>Accuracy of classification</th></tr>
<tr><td>Zero Crossing Rate</td><td><img src="Zero_Crossing_thresh_13000_acc_60.75.jpg" height=250px width=500px></td><td>60.75</td></tr>
<tr><td>Spectral Spread</td><td><img src="spectral_spread_thresh_10000_acc_54.5.jpg" height=250px width=500px></td><td>54.5</td></tr>
<tr><td>Spectral Rolloff</td><td><img src="Spectral_Rolloff_thresh_6000_acc_57.33.jpg" height=250px width=500px></td><td>57.33</td></tr>
<tr><td>Spectral Flux</td><td><img src="Spectral_flux_sdt_dev_thresh_0.03_acc_50.833.jpg" height=250px width=500px></td><td>50.833</td></tr>
<tr><td>Spectral Flatness</td><td><img src="Spectral_Flatness_thresh_1000_acc_61.25.jpg" height=250px width=500px></td><td>61.25</td></tr>
<tr><td>Spectral Centroid</td><td><img src="Spectral_Centroid_thresh_negative_6000_acc_58.08.jpg" height=250px width=500px></td><td>58.08</td></tr>
<tr><td>High Frequency Energy</td><td><img src="HF_Energy_thresh_13000_acc_58.03.jpg" height=250px width=500px></td><td>58.03</td></tr>
</table></p>
</div>
<div class="heading">Code</div>
<p>The code can be found <a href="https://github.com/unclegupta/DSP-PROJECT/blob/master/DSP_1D_Kalman_more_features.m">here</a></p>
</div>
<div class="section">
<div class="heading">4. Conclusions</div>
<div class="subsection">
<div class="heading">4.1 Summary</div>
<div class="text">
<!-- Start edit here -->
The results show that there is enough difference between advertising and music
that an algorithm that does not understand the content of the audio can still separate
them.
<br>The attributes chosen to define advertising in this project were all extracted from the audio
stream. This was not a necessity, but makes our application completely independent of outside
information. If we had relied on attributes such as the time, our application would by necessity
have required another information stream, independent of the audio stream.
<br>Thus the heavy reliance on attributes the algorithms failed in the following set of situations:
<ol>
<li>Any advertisement that consisted of music was classified as music. In essence
this is not wrong, but at the same time it meant that advertisements were wrongly classified.
</li>
<li>The DJ/Rap songs was consistently classified as advertising. This is something that cannot
be fixed by introducing new attributes. Although theoretically there are some
attributes that distinguish the DJ from advertising (such as the rhythm of speech), these are not very consistent,
and also very hard to classify.
</li>
<li>Some songs simply consisted of speech and music, exactly like advertising.At the same time some advertisements
had music included.
</li>
<ol>
<br>Due to the limited time available for our project, there are several steps still required
to make our product usable for the end user. Additionally we have not fully
explored every possible method of detecting advertising.
<!-- Stop edit here -->
</div>
</div>
<div class="subsection">
<div class="heading">4.2 Future Extensions</div>
<div class="text">
<!-- Start edit here -->
The success of the Kalman Filter could be extended further by passing more attributes to
it. Perhaps several of the attributes we are filtering provide more meaning when we
consider their change over time, instead of using their mean value every second.
At the moment only the volume levels are passed in as attributes, but this could
easily be extended by using vectors of several attributes.
<br>We can extend the concept to EKF (Extended Kalman Filter). The EKF can deal with non-limear systems and hence is expected to give better result. We can also use Dynamic Kalman Filter.
<!-- Stop edit here -->
</div>
</div>
</div>
</div>
</body>
</html>