Noise makes looking for single calls an unreliable strategy

[markgloverswaybtinternetcom]

As previous issue, noise is extending calls, thus being seen as the wrong species with high probability but wrong and sometimes extra species occur on the end of correctly recognised calls.

[scanjack]

I've noticed the same thing with recordings.
I wrote a batch file that analyzes different DT values ​​from BatDetect2.

My experience:
At a DT value of 0.3, very weak calls, but also background noise, are analyzed.
At a DT value of 0.5, weak calls and less background noise are taken into account.
At a DT value of 0.9, only strong and clear calls are analyzed.

Unfortunately, it's now winter, and here in Paderborn, Germany, there are hardly any bats flying anymore. So I'll have to wait a little longer with my field trials and tests.

Best regards

[markgloverswaybtinternetcom]

Hi Jochem, Tried rerunning recent post frost recordings on 0.5 detection threshold (I had to alter config.txt as it was over-riding my setting). Seems to get rid of the false bats and not loose the Barbastelle nose calls. With so few bats out, false calls was equalling real calls. Your idea of auto detection threshold is probably worth trying. Surprising bats only early on last night, 4 degrees later. I would be interested in a classifier including continental bats (I look at recording from Europe and can get more exotic species here in the South of England). Regards Mark Glover

…

------ Original Message ------ From "Mark Glover" ***@***.***> To "macaodha/batdetect2" ***@***.***> Date 29/11/2025 11:29:23 Subject Re[3]: [macaodha/batdetect2] Noise makes looking for single calls an unreliable strategy (Issue #50)

>Hi Jochem >I always stuck with the recommended threshold 0.3 but have noticed >lesser values at various stages within the batdetect2 code. >I do auto adjust the colour threshold on each GUI spectrogram section >but classification is across all wav files in a directory. >I have noticed batdetect2 flipping high confidence of one species to >another for the same bat (high, low to low, high), not the best >example, but I liked the feeding buzz (over water). >Obviously one can rarely be certain of species, but a single ID is >good for labels. Before I was displaying N calls from a table, but >always with time expanded sound. >I have had a very rare instance of many hundred ids in one second >which hung my gaming GUI. >Two short files attached recorded after the icy weather, first is miss >ID, train noise (full file gives spurious, cut down file for email >server does not), the second superb ID bat. > >I have not had the courage to try training classifiers. > Regards > Mark Glover > > >------ Original Message ------ >From "Jochen Roth" ***@***.***> >To "macaodha/batdetect2" ***@***.***> >Cc "markgloverswaybtinternetcom" ***@***.***>; >"Author" ***@***.***> >Date 27/11/2025 19:12:20 >Subject Re: [macaodha/batdetect2] Noise makes looking for single calls >an unreliable strategy (Issue #50) > >>scanjack left a comment (macaodha/batdetect2#50) >><#50 (comment)> >>I've noticed the same thing with recordings. >>I wrote a batch file that analyzes different DT values ​​from >>BatDetect2. >> >>My experience: >>At a DT value of 0.3, very weak calls, but also background noise, are >>analyzed. >>At a DT value of 0.5, weak calls and less background noise are taken >>into account. >>At a DT value of 0.9, only strong and clear calls are analyzed. >> >>Unfortunately, it's now winter, and here in Paderborn, Germany, there >>are hardly any bats flying anymore. So I'll have to wait a little >>longer with my field trials and tests. >> >>Best regards >> >>— >>Reply to this email directly, view it on GitHub >><#50 (comment)>, >>or unsubscribe >><https://github.com/notifications/unsubscribe-auth/AZP7OYBOU6JO5VB5UMMVQKL365EJJAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKOBXGA3DSMBTGU>. >>You are receiving this because you authored the thread.Message ID: >>***@***.***> >>

[scanjack]

Hi Mark,

Thank you very much for your valuable insights, especially the observation regarding the 0.5 detection threshold after the frost—that’s a crucial point for the accuracy of our evaluations. It's good to hear that this reduces the false positives without losing the Barbastelle nose calls.

Reply regarding Recording Technique and False Positives
You've understood my analysis correctly: Yes, my TeensyBat system uses the Full Spectrum (FS) method for recording.

This is exactly why I tend to experience the problems you described (high false positives and "flipping" confidence) less often, or can solve them differently:

Time Expansion (TE) vs. Full Spectrum (FS): You mentioned using TE recordings (Time Expanded Sound) in the past. While the TE mode can make hearing easier, the digital delay and processing can distort the noise and the spectral features of the calls (which the classifiers rely on). My TeensyBat records the Full Spectrum (FS), which results in an unadulterated raw file, ideal for neural networks like BatDetect2.

False Positives and Noise: When BatDetect2 is applied to heavily noisy TE recordings (or generally to loud sections like train noise), the model can interpret lower detection thresholds (like 0.3) as a real call. Raising it to 0.5, as you did, is the correct countermeasure for such noisy datasets.

My Solution (Time and Environment): Thanks to the FS method and the linkage to my Nexus Logger (which uses a GPS Timefix), I can correlate every call to the millisecond with the environmental data (temperature, pressure, humidity). This allows me to filter false positives based not only on confidence but also on external conditions (e.g., strong wind).

Interest in Continental Species
I wholeheartedly share your interest in a classifier for continental European species. My primary research area is in Central Europe, and the accurate identification of rarer species is crucial for our project.

Your observations are very valuable. Let's stay in touch, especially if you have more data on BatDetect2's performance with Central European species.

Best regards,

Jochen

[markgloverswaybtinternetcom]

Hi Jochen, Sound for Humans I think you miss understood my use of Time Expansion, Neil Middleton teaches listening, as well spectrogram full spectrum visual. This why I use a gaming GUI, it has a 5 ms update time which allows me to hear and have a cursor on the spectrogram and move quickly through the recordings (arrow keys: up down = files, left right = time), cancelling previous less interesting sounds. Also TE sound is slightly less alien to newbies and more portable (I also increase the amplitude by 10). I do not allow the classifier to read TE sound. Species Probabilities My point about flipping, was because I looked at the species probabilities, to see if there was an easy fix. Because of the drastic change in species probabilities for a small change in calls of the same bat I decided there was no easy fix at a high level, that it needed fixing at a low level. Sonobat seems to work on probability a lot but I find its minimum and maximum values are slightly too tight, producing similar problems. GPS recordings I occasionally use a Wildlife Acoustics Echo Meter 2 when walking. As the classifier on this is poor I put it into BatDetect2 and then manually check it and my GUI produces a HTML open street map of the bat species, with a hyper-link to the TE recording. But I am not a late night person, so my recording mainly comes from static recorders. Regards Mark Glover

…

------ Original Message ------ From "Jochen Roth" ***@***.***> To "macaodha/batdetect2" ***@***.***> Cc "markgloverswaybtinternetcom" ***@***.***>; "Author" ***@***.***> Date 30/11/2025 21:45:45 Subject Re: [macaodha/batdetect2] Noise makes looking for single calls an unreliable strategy (Issue #50)

scanjack left a comment (macaodha/batdetect2#50) <#50 (comment)> Hi Mark, Thank you very much for your valuable insights, especially the observation regarding the 0.5 detection threshold after the frost—that’s a crucial point for the accuracy of our evaluations. It's good to hear that this reduces the false positives without losing the Barbastelle nose calls. Reply regarding Recording Technique and False Positives You've understood my analysis correctly: Yes, my TeensyBat system uses the Full Spectrum (FS) method for recording. This is exactly why I tend to experience the problems you described (high false positives and "flipping" confidence) less often, or can solve them differently: Time Expansion (TE) vs. Full Spectrum (FS): You mentioned using TE recordings (Time Expanded Sound) in the past. While the TE mode can make hearing easier, the digital delay and processing can distort the noise and the spectral features of the calls (which the classifiers rely on). My TeensyBat records the Full Spectrum (FS), which results in an unadulterated raw file, ideal for neural networks like BatDetect2. False Positives and Noise: When BatDetect2 is applied to heavily noisy TE recordings (or generally to loud sections like train noise), the model can interpret lower detection thresholds (like 0.3) as a real call. Raising it to 0.5, as you did, is the correct countermeasure for such noisy datasets. My Solution (Time and Environment): Thanks to the FS method and the linkage to my Nexus Logger (which uses a GPS Timefix), I can correlate every call to the millisecond with the environmental data (temperature, pressure, humidity). This allows me to filter false positives based not only on confidence but also on external conditions (e.g., strong wind). Interest in Continental Species I wholeheartedly share your interest in a classifier for continental European species. My primary research area is in Central Europe, and the accurate identification of rarer species is crucial for our project. Your observations are very valuable. Let's stay in touch, especially if you have more data on BatDetect2's performance with Central European species. Best regards, Jochen — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZP7OYB36CI2AEIFM2SJ2FD37NQQTAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKOJTGU2DCMRVGM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[chrmue44]

Hi Jochen and Mark,
I'm currently working on a batdetect2 model to detect all 25 species occurring in Germany. It is not finished yet because I am still searching for more training data.
But at least it is able to detect more species than the original model.
Below you can see what I have so far for training.
The detection of species with less than lets say 500 calls is unstable.
For the moment I removed the species Vespertilio murinus from the model because I get many false positives on Nyctalus noctula calls.

Species	Calls
Barbastella barbastellus	2475
Eptesicus nilssonii	1083
Eptesicus serotinus	1352
Hypsugo savii	429
Myotis alcathoe	292
Myotis bechsteinii	787
Myotis brandtii	517
Myotis dasycneme	1778
Myotis daubentonii	3836
Myotis emarginatus	663
Myotis myotis	2586
Myotis mystacinus	1929
Myotis natteri	2364
Nyctalus leisleri	1476
Nyctalus noctula	2715
Pipistrellus kuhlii	2591
Pipistrellus nathusii	2620
Pipistrellus pipistrellus	3438
Pipistrellus pygmaeus	4408
Plecotus auritus	461
Plecotus austriacus	661
Rhinolophus hipposideros	110
Rhinolophus ferrumequinum	823

I have some more recordings of Plecotus, Rhinolophus hipposideros and Myotis myotis that are not annotated yet, so I expect some slight improvements in the near future.

Let me know if you are interested. Perhaps you could even provide some training data?

Cheers
Christian

[scanjack]

Hi Christian,

This is truly fantastic news!

As someone who researches bat calls and uses BatDetect2 for analysis, I'm very interested in a model tailored to the 25 German species. This would significantly improve the accuracy of the analysis I'm aiming for.

It's understandable that the detection of species with fewer than 500 calls is unstable. The challenge with Vespertilio murinus and Nyctalus noctula demonstrates how important a broad and clean data set is to address these problems at a low level.

Unfortunately, my own data set is currently very limited, as my self-developed project Nexus records environmental data, but I use BatDetect2 for analyzing bat calls.

However, I do have data from the Animal Sound Archive and CHIROVOX. The animal sounds archive is freely accessible (link: https://suche.tierstimmenarchiv.de/), but registration is required for CHIROVOX. I would need to check to what extent this data may be shared for training purposes, but perhaps the animal sounds archive is of direct interest to you.

Please let me know what your requirements are for the training data (e.g., file format, annotation standard, or whether raw data is required).

Best regards,
Jochen

[chrmue44]

Hi Jochen, thanks for the hints, but I already scraped off everything I could from those two sources. Should I post my intermediate model here? We are somehow hijacking this thread... Cheers Christian Jochen Roth ***@***.***> schrieb am Di., 2. Dez. 2025, 11:42:

…

*scanjack* left a comment (macaodha/batdetect2#50) <#50 (comment)> Hi Christian, This is truly fantastic news! As someone who researches bat calls and uses BatDetect2 for analysis, I'm very interested in a model tailored to the 25 German species. This would significantly improve the accuracy of the analysis I'm aiming for. It's understandable that the detection of species with fewer than 500 calls is unstable. The challenge with Vespertilio murinus and Nyctalus noctula demonstrates how important a broad and clean data set is to address these problems at a low level. Unfortunately, my own data set is currently very limited, as my self-developed project Nexus records environmental data, but I use BatDetect2 for analyzing bat calls. However, I do have data from the Animal Sound Archive and CHIROVOX. The animal sounds archive is freely accessible (link: https://suche.tierstimmenarchiv.de/), but registration is required for CHIROVOX. I would need to check to what extent this data may be shared for training purposes, but perhaps the animal sounds archive is of direct interest to you. Please let me know what your requirements are for the training data (e.g., file format, annotation standard, or whether raw data is required). Best regards, Jochen — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJNOSH573QWWP3G7B2XHBD37VUJDAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMBRGM3TMMBUGY> . You are receiving this because you commented.Message ID: ***@***.***>

[markgloverswaybtinternetcom]

Hi Christians labelling effort sounds really impressive, I am out of my depth. Sort of thing that I would get to the end of and realise what I should have been doing at the start. CHIROVOX looks good, I can access Bat Conservation Trust that it mentions but they are more limited but they do mention cluttered / semi-cluttered / open habitat One thing that did come up in a recent conversation with the BTO (cloud autoID) is the importance negative samples like cars, trains, crickets etc.. If there were lots of samples, social call, small bat social calls are big bats in batdetect2. Some social calls are more unique than the echolocations but most need the echolocation to id. Regards Mark Glover

…

------ Original Message ------ From "Christian Müller" ***@***.***> To "macaodha/batdetect2" ***@***.***> Cc "markgloverswaybtinternetcom" ***@***.***>; "Author" ***@***.***> Date 02/12/2025 15:47:14 Subject Re: [macaodha/batdetect2] Noise makes looking for single calls an unreliable strategy (Issue #50)

chrmue44 left a comment (macaodha/batdetect2#50) <#50 (comment)> Hi Jochen, thanks for the hints, but I already scraped off everything I could from those two sources. Should I post my intermediate model here? We are somehow hijacking this thread... Cheers Christian Jochen Roth ***@***.***> schrieb am Di., 2. Dez. 2025, 11:42: > *scanjack* left a comment (macaodha/batdetect2#50) > <#50 (comment)> > > Hi Christian, > > This is truly fantastic news! > > As someone who researches bat calls and uses BatDetect2 for analysis, I'm > very interested in a model tailored to the 25 German species. This would > significantly improve the accuracy of the analysis I'm aiming for. > > It's understandable that the detection of species with fewer than 500 > calls is unstable. The challenge with Vespertilio murinus and Nyctalus > noctula demonstrates how important a broad and clean data set is to address > these problems at a low level. > > Unfortunately, my own data set is currently very limited, as my > self-developed project Nexus records environmental data, but I use > BatDetect2 for analyzing bat calls. > > However, I do have data from the Animal Sound Archive and CHIROVOX. The > animal sounds archive is freely accessible (link: > https://suche.tierstimmenarchiv.de/), but registration is required for > CHIROVOX. I would need to check to what extent this data may be shared for > training purposes, but perhaps the animal sounds archive is of direct > interest to you. > > Please let me know what your requirements are for the training data (e.g., > file format, annotation standard, or whether raw data is required). > > Best regards, > Jochen > > — > Reply to this email directly, view it on GitHub > <#50 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AEJNOSH573QWWP3G7B2XHBD37VUJDAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMBRGM3TMMBUGY> > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZP7OYEFOGNPS2LHEBVJS2T37WYAFAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMBSG4YDQMRSGY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[chrmue44]

Hi Mark,
yes I also added some randam noise like crickets, rain and other stuff to the training data.
Please give me some time to release the model. There are more recordings to annotate and then I have to retrain the model and evaluate wether it got better or not.

A big help would be more recordings especially of the rare species. So if you could get access please let me know!

There is a nice freely available validation data at batident.eu that I use to test the model

Best regards
Christian

[scanjack]

Hello Christian and Mark,
Thank you so much for your valuable comments and the discussion – Christian's model development for the 25 German species is truly impressive.

Mark, I completely agree with you that negative examples are essential, and Christian, it's great that you've already included noises like crickets and rain!
In my own recordings with the TeensyBat, I'm addressing the problem of low-frequency background noise technically: I'm using a high-pass filter (HPF) that filters out all frequencies below 15-16 kHz. This prevents much of the low-frequency noise (cars, wind, etc.) from even entering the analysis with BatDetect2.
I'm currently trying to solve the problem of distinguishing between social calls and echolocation (and the general accuracy I'm aiming for) as follows: I run BatDetect2 three times with different detection thresholds (DTs):
DT 0.3 for social calls.
DT 0.5 for feeding buzzes.
DT 0.9 for the pure echolocation calls.
The overlap of the results from these runs gives me good validation confidence and helps to better pinpoint uncertain or misclassified signals.

Christian, please take your time with the careful annotation and retraining.
My own database is sparse (my Nexus only records environmental data, not the calls), but I would be happy to support you.
You marked the species with fewer than 500 calls as "unstable." Which of the rare species are currently the highest priority for you? It would be helpful to know which specific species you need most urgently so that I can specifically look for suitable sources or my own recordings.
Best regards,
Jochen

[markgloverswaybtinternetcom]

Hi Christian Rare Recordings I have samples from 3 books: European bats, Social calls of UK bats, 'Is That a Bat' plus UK bat acoustics technician course. However being certain of rare bats often means handling them. Classification work-flow I batch classify everything and display the labelled plots. Classification for me is about saving effort without throwing information away. So I would prefer that the classifier is able to adjust its own levels But I display everything, just use the first consecutive labels as the starting point, and occasionally go to highest probability of other species. BatDetect2 rarely handles social and feeding buzzes currently. Smaller bat feeding buzzes are just on the resolution limits of spectrograms. Filtering I filter for display purposes only (15 kHz high pass default) not classification, I was not seeing any false classification in this region. I see reports of bats below this frequency hence it is adjustable. OpenSoundScape has been using band pass filtering it has advantages and disadvantages. Humans have limited colour resolution, whereas classifiers see numbers. Regards Mark Glover

…

------ Original Message ------ From "Jochen Roth" ***@***.***> To "macaodha/batdetect2" ***@***.***> Cc "markgloverswaybtinternetcom" ***@***.***>; "Author" ***@***.***> Date 03/12/2025 22:01:58 Subject Re: [macaodha/batdetect2] Noise makes looking for single calls an unreliable strategy (Issue #50)

scanjack left a comment (macaodha/batdetect2#50) <#50 (comment)> Hello Christian and Mark, Thank you so much for your valuable comments and the discussion – Christian's model development for the 25 German species is truly impressive. Mark, I completely agree with you that negative examples are essential, and Christian, it's great that you've already included noises like crickets and rain! In my own recordings with the TeensyBat, I'm addressing the problem of low-frequency background noise technically: I'm using a high-pass filter (HPF) that filters out all frequencies below 15-16 kHz. This prevents much of the low-frequency noise (cars, wind, etc.) from even entering the analysis with BatDetect2. I'm currently trying to solve the problem of distinguishing between social calls and echolocation (and the general accuracy I'm aiming for) as follows: I run BatDetect2 three times with different detection thresholds (DTs): DT 0.3 for social calls. DT 0.5 for feeding buzzes. DT 0.9 for the pure echolocation calls. The overlap of the results from these runs gives me good validation confidence and helps to better pinpoint uncertain or misclassified signals. Christian, please take your time with the careful annotation and retraining. My own database is sparse (my Nexus only records environmental data, not the calls), but I would be happy to support you. You marked the species with fewer than 500 calls as "unstable." Which of the rare species are currently the highest priority for you? It would be helpful to know which specific species you need most urgently so that I can specifically look for suitable sources or my own recordings. Best regards, Jochen — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZP7OYB673PQVWIRNFT3ZP3375MVNAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMBZGAZDOMBVG4>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[chrmue44]

Batdetect2 for UK species is not yet trained to detect social calls or feeding buzzes, my model neither! (see paper)
The detection of social calls would require much more of carefully selected training data that is not available to my knowledge.

Remarks about training data:
I need at least around 100 clips (1 .. 5 seconds) per species.
Before using randomly collected recordings, we have to clarify if we are allowed to use them for our purpose (copyright).

Cheers
Christian

[markgloverswaybtinternetcom]

Hi Christian, Feeding Buzzes As I have said previously individual calls that we could get may hundreds of are useless. The key to recognising feeding buzzes is the repetition rate, Sonobat uses a separate program to recognise feeding buzzes. Social Calls Quite a lot of the social calls are multi-component but not as badly as feeding buzzes. The other problem is that there are more than 4 categories of social call. I would have enough if it was not for this. I think we could simplify this to the most common C, D calls Regards Mark Glover

…

------ Original Message ------ From "Christian Müller" ***@***.***> To "macaodha/batdetect2" ***@***.***> Cc "markgloverswaybtinternetcom" ***@***.***>; "Author" ***@***.***> Date 04/12/2025 12:06:44 Subject Re: [macaodha/batdetect2] Noise makes looking for single calls an unreliable strategy (Issue #50)

chrmue44 left a comment (macaodha/batdetect2#50) <#50 (comment)> Batdetect2 for UK species is not yet trained to detect social calls or feeding buzzes, my model neither! (see paper) The detection of social calls would require much more of carefully selected training data that is not available to my knowledge. Remarks about training data: I need at least around 100 clips (1 .. 5 seconds) per species. Before using randomly collected recordings, we have to clarify if we are allowed to use them for our purpose (copyright). Cheers Christian — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZP7OYEESA34Z4VFWBLX7ST4AAPVJAVCNFSM6AAAAAB4647Z2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMJRHA4TSNBXGY>. You are receiving this because you authored the thread.Message ID: ***@***.***>