API: support modules

Frequency tracking

Even though the spectrogram is one of the most dominant time-frequency representation, there are whole class of alternate representations. This module has the code which tracks the dominant frequency in a sound using non-spectrogram methods.

The Pseudo Wigner Ville Distribution

The Pseudo Wigner Ville Distribution is an accurate but not so well known method to represent a signal on the time-frequency axis[1]. This time-frequency representation is implemented in the get_pwvd_frequency_profile.

References

[1] Cohen, L. (1995). Time-frequency analysis (Vol. 778). Prentice hall.

itsfm.frequency_tracking.get_pwvd_frequency_profile(input_signal, fs, **kwargs)

Generates a clean frequency profile through the PWVD. The order of frequency profile processing is as follows:

1. Split input signal into regions that are greater or equal to the signal_level. This speeds up the whole process of pwvd tracking multiple sounds, and ignores the fainter samples.

2. Generate PWVD for each above-noise region.

3. Set regions below background noise to 0Hz

4. Remove sudden spikes and set these regions to values decided by interpolation between adjacent non-spike regions.

Parameters

• input_signal (np.array) –

• fs (float) –

Notes

The fact that each signal part is split into independent above-background segments and then frequency tracked can have implications for frequency resolution. Short sounds may end up with frequency profiles that have a lower resolution than longer sounds. Each sound is handled separately primarily for memory and speed considerations.

Example

Create two chirps in the middle of a somewhat silent recording

>>> import matplotlib.pyplot as plt
>>> from itsfm.simulate_calls import make_fm_chirp
>>> from itsfm.view_horseshoebat_call import plot_movingdbrms
>>> from itsfm.view_horseshoebat_call import visualise_call, make_x_time
>>> fs = 44100
>>> start_f, end_f = 1000, 10000
>>> chirp = make_fm_chirp(start_f, end_f, 0.01, fs)
>>> rec = np.random.normal(0,10**(-50/20), 22100)
>>> chirp1_start, chirp1_end = 10000, 10000 + chirp.size
>>> chirp2_start, chirp2_end = np.array([chirp1_start, chirp1_end])+int(fs*0.05)
>>> rec[chirp_start:chirp_end] += chirp
>>> rec[chirp2_start:chirp2_end] += chirp
>>> rec /= np.max(abs(rec))
>>> actual_fp = np.zeros(rec.size)
>>> actual_fp[chirp1_start:chirp1_end] = np.linspace(start_f, end_f, chirp.size)
>>> actual_fp[chirp2_start:chirp2_end] = np.linspace(start_f, end_f, chirp.size)

Check out the dB rms profile of the recording to figure out where the noise floor is

>>> plot_movingdbrms(rec, fs)

>>> clean_fp, info = get_pwvd_frequency_profile(rec, fs,
                                                     signal_level=-9,
                                                     extrap_window=10**-3,
                                                     max_acc = 0.6)
>>> plt.plot(clean_fp, label='obtained')
>>> plt.plot(actual_fp, label='actual')
>>> plt.legend()

Now, let’s overlay the obtained frequency profile onto a spectrogram to check once more how well the dominant frequency has been tracked.

>>> w,s = visualise_call(rec, fs, fft_size=128)
>>> s.plot(make_x_time(clean_fp, fs), clean_fp)

See also

itsfm.signal_cleaning.smooth_over_potholes(), find_above_noise_regions()

itsfm.frequency_tracking.find_geq_signallevel(X, fs, **kwargs)

Find regions greater or equal to signal level

itsfm.frequency_tracking.clean_up_spikes(whole_freqeuncy_profile, fs, **kwargs)

Applies smooth_over_potholes on each non-zero frequency segment

in the profile.

smooth_over_potholes

Let’s create a case with an FM and CF tone

>>> from itsfm.simulate_calls import make_tone, make_fm_chirp, silence
  >>> fs = 22100
  >>> tone = make_tone(5000, 0.01, fs)
  >>> sweep = make_fm_chirp(1000, 6000, 0.005, fs)
  >>> gap = silence(0.005, fs)
  >>> full_call = np.concatenate((tone, gap, sweep))

The raw frequency profile, with very noisy frequency estimates needs to be further cleaned

>>> raw_fp, frequency_index = generate_pwvd_frequency_profile(full_call,
                                                                fs)
>>> noise_supp_fp = noise_supp_fp = suppress_background_noise(raw_fp,
                                          full_call,
                                          window_size=25,
                                          background_noise=-30)

Even after the noisy parts have been suppressed, there’re still some spikes caused by the

>>>

itsfm.frequency_tracking.generate_pwvd_frequency_profile(input_signal, fs, **kwargs)

Generates the raw instantaneous frequency estimate at each sample. using the Pseudo Wigner Ville Distribution

Parameters

• input_signal (np.array) –

• fs (float) –

• pwvd_filter (Boolean, optional) – Whether to perform median filtering with a 2D kernel. Defaults to False

• pwvd_filter_size (int, optional) – The size of the square 2D kernel used to median filter the initial PWVD time-frequency representation.

• pwvd_window (float>0, optional) – The duration of the window used in the PWVD. See pwvd_transform for the default value.

• tfr_cliprange (float >0, optional) – The clip range in dB. Clips all values in the abs(pwvd) time-frequency representation to between max and max*10*(-tfr_cliprange/20.0). Defaults to None, which does not alter the pwvd transform in anyway.

Returns

raw_frequency_profile, frequency_indx – Both outputs are the same size as input_signal. raw_frequency_profile is the inst. frequency in Hz. frequency_indx is the row index of the PWVD array.

Return type

np.array

See also

pwvd_transform(), track_peak_frequency_over_time(), itsfm.signal_cleaning.clip_tfr()

itsfm.frequency_tracking.pwvd_transform(input_signal, fs, **kwargs)

Converts the input signal into an analytical signal and then generates the PWVD of the analytical signal.

Uses the PseudoWignerVilleDistribution class from the tftb package [1].

Parameters

• input_signal (np.array) –

• fs (float) –

• pwvd_window_type (np.array, optional) – The window to be used for the pseudo wigner-ville distribution. If not given, then a hanning signal is used of the default length. The window given here supercedes the ‘window_length’ argument below.

• pwvd_window (float>0, optional) – The duration of the window used in the PWVD. Defaults to 0.001s

Returns

time_frequency_output – Two dimensional array with dimensions of NsamplesxNsamples, where Nsamples is the number of samples in input_signal.

Return type

np.array

References

[1] Jaidev Deshpande, tftb 0.1.1 ,Python module for time-frequency analysis,

https://pypi.org/project/tftb/

itsfm.frequency_tracking.track_peak_frequency_over_time(input_signal, fs, time_freq_rep, **kwargs)

Tracks the lowest possible peak frequency. This ensures that the lowest harmonic is being tracked in a multiharmonic signal with similar levels across the harmonics.

EAch ‘column’ of the 2D PWVD is inspected for the lowest peak that crosses a percentile threshold, and this is then taken as the peak frequency.

Parameters

• input_signal (np.array) –

• fs (float>0) –

• time_freq_rep (np.array) – 2D array with the PWVD representation.

• percentile (0<float<100, optional) –

Returns

peak_freqs, peak_inds – Arrays with same size as the input_signal. peak_freqs is the frequencies in Hz, peak_inds is the row index.

Return type

np.array

See also

find_lowest_intense_harmonic_across_TFR(), get_most_intense_harmonic()

itsfm.frequency_tracking.find_lowest_intense_harmonic_across_TFR(tf_representation, **kwargs)

itsfm.frequency_tracking.get_most_intense_harmonic(time_slice, **kwargs)

Searches a single column in a 2D array for the first region which crosses the given percentile threshold.

itsfm.frequency_tracking.get_midpoint_of_a_region(region_object)

itsfm.frequency_tracking.accelaration(X, fs)

Calculates the absolute accelrateion of a frequency profile in kHz/ms^2

itsfm.frequency_tracking.speed(X, fs)

Calculates the abs speed of the frequency profile in kHz/ms

itsfm.frequency_tracking.get_first_region_above_threshold(input_signal, **kwargs)

Takes in a 1D signal expecting a few peaks in it above the percentil threshold. If all samples are of the same value, the region is restricted to the first two samples.

Parameters

• input_signal (np.array) –

• percentile (0<float<100, optional) – The percentile threshold used to set the threshold. Defaults to 99.5

Returns

region_location – If there is at least one region above the threshold a tuple with the output from scipy.ndimage.find_objects. Otherwise None.

Return type

tuple or None

itsfm.frequency_tracking.frequency_spike_detection(X, fs, **kwargs)

Detects spikes in the frequency profile by monitoring the accelration profile through the sound.

Parameters

• X (np.array) – A frequency profile with sample-level estimates of frequency in Hz

• fs (float>0) –

• max_acc (float>0, optional) – Maximum acceleration in the frequency profile. Defaults to 0.5kHz/ms^2

Returns

anomalous – Boolean

Return type

np.array

Signal processing

Functions which actually do the calculations on the raw input signal

Module with signal processing functions in it used by both measure and segment modules.

itsfm.signal_processing.dB(X)

Calculates the 20log of X

itsfm.signal_processing.rms(X)

Root mean square of a signal

itsfm.signal_processing.calc_energy(X)

Sum of all squared samples

itsfm.signal_processing.get_power_spectrum(audio, fs=250000.0)

Calculates an RFFT of the audio. :param audio: :type audio: np.array :param fs: Frequency of sampling in Hz :type fs: int

Returns

• dB_power_spectrum (np.array) – dB(power_spectrum)

• freqs (np.array) – Centre frequencies of the RFFT.

itsfm.signal_processing.calc_sound_borders(audio, percentile=99)

Gives the start and stop of a sound based on the percentile cumulative energy values.

Parameters

• audio (np.array) –

• percentile (float, optional) – Value between 100 and 0. The sound border is calcualted as the border which encapsulates the percentile of energy Defaults to 99.

Returns

start, end

Return type

int

itsfm.signal_processing.get_robust_peak_frequency(audio, **kwargs)

Makes a spectrogram from the audio and calcualtes the peak frequency by averaging each slice of the spectrogram’s FFT’s.

This ‘smooths’ out the structure of the power spectrum and allows a single and clear peak detection.

Thanks to Holger Goerlitz for the suggestion.

Parameters

• audio (np.array) –

• fs (float) – Frequency of sampling in Hz

• seg_length (int, optional) – The size of the FFt window used to calculate the moving FFT slices. DEfaults to 256

• noverlap (int, optional) – The number of samples overlapping between one FFT slice and the next. Defaults to seg_length -1

Returns

peak_frequency – Frequency with highest power in the audio in Hz.

Return type

float

itsfm.signal_processing.get_peak_frequency(audio, fs)

Gives peak frequency and frequency resolution with which the measurement is made

Parameters

• audio (np.array) –

• fs (float>0) – sampling rate in Hz

Returns

peak_freq, freq_resolution – The peak frequency and frequency resolution of this peak frequency in Hz.

Return type

float

itsfm.signal_processing.get_frequency_resolution(audio, fs)

Parameters

• audio (np.array) –

• fs (float>0) – sampling rate in Hz

Returns

resolution – The frequency resolution in Hz.

Return type

float

itsfm.signal_processing.moving_rms(X, **kwargs)

Calculates moving rms of a signal with given window size. Outputs np.array of same size as X. The rms of the last few samples <= window_size away from the end are assigned to last full-window rms calculated

Parameters

• X (np.array) – Signal of interest.

• window_size (int, optional) – Defaults to 125 samples.

Returns

all_rms – Moving rms of the signal.

Return type

np.array

itsfm.signal_processing.moving_rms_edge_robust(X, **kwargs)

Calculates moving rms of a signal with given window size. Outputs np.array of same size as X. This version is robust and doesn’t suffer from edge effects as it calculates the moving rms in both forward and backward directions and calculates a consensus moving rms profile.

The consensus rms profile is basically achieved by taking the left half of the forward rms profile and concatenating it with the right hald of the backward passed rms profile.

Parameters

• X (np.array) – Signal of interest.

• window_size (int, optional) – Defaults to 125 samples.

Returns

all_rms – Moving rms of the signal.

Return type

np.array

Notes

moving_rms_edge_robust may not be too accurate when the rms is expected to vary over short time scales in the centre of the signal!!

itsfm.signal_processing.form_consensus_moving_rms(forward, backward)

Parameters

• backward (forward,) – Two arrays of the same dimensions.

• and returns the consensus maximum value at each sample. (Compares) –

itsfm.signal_processing.median_filter(input_signal, fs, **kwargs)

Median filters a signal according to a user-settable window size.

Parameters

• input_signal (np.array) –

• fs (float) – Sampling rate in Hz.

• medianfilter_size (float, optional) – The window size in seconds. Defaults to 0.001 seconds.

Returns

med_filtered – Median filtered version of the input_signal.

Return type

np.array

itsfm.signal_processing.calc_proper_kernel_size(durn, fs)

scipy.signal.medfilt requires an odd number of samples as kernel_size. This function calculates the number of samples for a given duration which is odd and is close to the required duration.

Parameters

• durn (float) – Duration in seconds.

• fs (float) – Sampling rate in Hz

Returns

samples – Number of odd samples that is equal to or little less (by one sample) than the input duration.

Return type

int

itsfm.signal_processing.resize_by_adding_one_sample(input_signal, original_signal, **kwargs)

Resizes the input_signal to the same size as the original signal by repeating one sample value. The sample value can either the last or the first sample of the input_signal.

itsfm.signal_processing.get_terminal_frequency(audio, fs, **kwargs)

Gives the -XdB frequency from the peak.

The power spectrum is calculated and smoothened over 3 frequency bands to remove complex comb-like structures.

Then the lowest frequency below XdB from the peak is returned.

Parameters

• audio (np.array) –

• fs (float>0) – Sampling rate in Hz

• terminal_frequency_threshold (float, optional) – The terminal frequency is calculated based on finding the level of the peak frequency and choosing the lowest frequency which is -10 dB (20log10) below the peak level. Defaults to -10 dB

Returns

• terminal_frequency

• threshold

Notes

Careful about setting threshold too low - it might lead to output of terminal frequencies that are actually in the noise, and not part of the signal itself.

Signal cleaning

Functions which refine, clean and detect outliers.

This module handles the identification and cleaning of noise in signals. A ‘noisy’ signal is one that has spikes in it or sudden variations in a continuous looking function. Most of these functions are built to detect and handle sudden spikes in the frequency profile estimates of a sound.

itsfm.signal_cleaning.exterpolate_over_anomalies(X, fs, anomalous, **kwargs)

Ex(tra)+(in)ter-polates –> Exterpolates over anomalous regions. Anomalous regions are either ‘edge’ or ‘island’ types. The ‘edge’ anomalies are those which are at the extreme ends of the signal. The ‘island’ anomalies are regions with non-anomalous regions on the left and right.

An ‘edge’ anomalous region is handled by running a linear regression on the neighbouring non-anomalous region, and using the slope to extrapolate over the edge anomaly.

An ‘island’ anomaly is handled by interpolating between the end values of the neighbouring non-anomalous regions.

Parameters

• X (np.array) –

• fs (float>0) – Sampling rate in Hz

• anomalous (np.array) – Boolean array of same size as X True indicates an anomalous sample.

• extrap_window (float>0, optional) – The duration of the extrapolation window in seconds. Defaults to 0.1ms

Returns

smooth_X – Same size as X, with the anomalous regions

Return type

np.array

Notes

Only extrapolation by linear regression is supported currently. The extrap_window parameter is important especially if there is a high rate of frequency modulation towards the edges of the sound. When there is a high freq. mod. at the edges it is better to set the extrap_window small. However, setting it too small also means that the extrapolation may not be as nice anymore.

Example

not up to date!!!

See also

find_closest_normal_region()

itsfm.signal_cleaning.fix_island_anomaly(X, fs, anomaly, ref_region_length, **kwargs)

First tries to interpolate between the edges of the anomaly at hand. If the interpolation leads to a very drastic slope, a ‘sensible’ extrapolation is attempted using parts of the non-anomalous signal.

Parameters

• X (np.array) –

• fs (float>0) –

• anomaly (tuple slice) – scipy.ndimage.find_objects output (slice(start,stop,None),)

• ref_region_length (int>0) – The number of samples to be used as a reference region in case of extrapolation

• max_fmrate (float>0, optional) – The maximum fm rate to be tolerated while interpolating in kHz/ms Defaults to 100 kHz/ms.

Returns

interpolated – Array of same size as anomaly.

Return type

np.array

itsfm.signal_cleaning.extrapolate_sensibly(X, fs, anomaly, ref_region_length, **kwargs)

Function called when fix_island_anomaly detects direct interpolation will lead to unrealistic slopes. This function is called when there’s a big difference in values across an anomalous region and an extrapolation must be performed which will not alter the signal drastically.

The method tries out the following:

1. Look left and right of the anomaly to see which region has higher frequency content.

2. Extrapolate in the high-to-low frequency direction.

This basically means that if the local inspection window around anomaly has a sweep between 20-10kHZ on the left and a 0Hz region on the right - the anomaly will be extrapolated with the slope from the sweep region because it has higher frequency content.

Example

>>> freq_profile = [np.zeros(10), np.arange(15,30,5)*1000]
>>> fs = 1.0
>>> x = np.concatenate(freq_profile)[::-1]
>>> anom = (slice(2, 5, None),)
>>>
>>> plt.plot(x, label='noisy frequency profile')
>>> anom_x = np.zeros(x.size, dtype='bool')
>>> anom_x[anom[0]] = True
>>> plt.plot(anom_x*8000, label='identified anomaly')
>>> extrap_out = extrapolate_sensibly(x, fs, anom, 4)
>>> sensibly_extrap = x.copy()
>>> sensibly_extrap[anom_x] = extrap_out
>>> plt.plot(sensibly_extrap, label='extrapolated')
>>> plt.legend()

itsfm.signal_cleaning.get_neighbouring_regions(X, target, region_size)

Takes out samples of region_size on either size of the target.

Parameters

• X (np.array) –

• target (slice) – ndimage.find_objects type slice

• region_size (int >0) –

Returns

left_and_right

Return type

list

itsfm.signal_cleaning.calc_coarse_fmrate(X, fs, **kwargs)

Calculates slope by subtracting the difference between 1st and last sample and dividing it by the length of the array. The output is then converted to units of kHz/ms.

Parameters

• X (np.array) – Frequency profile with values in Hz.

• fs (float>0) –

itsfm.signal_cleaning.anomaly_extrapolation(region, X, num_samples, **kwargs)

Takes X values next to the region and fits a linear regression into the region. This is only suitable for cases where the anomalous region is at an ‘edge’ - either one of its samples is 0 or the last sample of X.

Parameters

• region (object tuple) – A slice type object which is the output from scipy.ndimage.find_objects This is a slice inside a list/tuple.

• X (np.array) – The original array over which the extrapolation is to be performed

• num_samples (int>0) – The number of samples next to the region to be used to fit the data for extrapolation into the region.

Returns

extrapolated – The values corresponding to the extrapolated region.

Return type

np.array

Notes

1. This function covers 90% of cases…if there is an anomaly right next to an edge anomaly with <num_samples distance – of course things will go whack.

Warning

A mod on this function also allows extrapolation to occur if there are < num_samples next to the anomaly - this might make the function a bit lax in terms of the extrapolations it produces.

itsfm.signal_cleaning.anomaly_interpolation(region, X, **kwargs)

Interpolates X values using values of X adjacent to the region.

Parameters

• region (object tuple) – Output from scipy.ndimage.find_objects

• X (np.array) –

Returns

full_span – The values of interpolated X, of same size as the region length.

Return type

np.array

itsfm.signal_cleaning.smooth_over_potholes(X, fs, **kwargs)

A signal can show drastic changes in its value because of measurement errors. These drastic variations in signal are called potholes (uneven parts of a road). This method tries to ‘level’ out the pothole by re-setting the samples of the pothole. A linear interpolation is done from the start of a pothole till its end using the closest non-pothole samples.

A pothole is identified by a region of the signal with drastic changes in slope. A moving window calculates N slopes between the focal sample and the Nth sample after it to estimate if the Nth sample could be part of a pothole or not.

Parameters

• X (np.array) –

• fs (float>0) –

• max_stepsize (float>0, optional) – The maximum absolute difference between adjacent samples. Defaults to 50.

• pothole_inspection_window (float>0, optional) – The length of the moving window that’s used to discover potholes. See identify_pothole_samples for default value.

Returns

• pothole_covered

• pothole_regions

See also

identify_pothole_samples(), pothole_inspection_window()

itsfm.signal_cleaning.identify_pothole_samples(X, fs, **kwargs)

Moves a sliding window and checks the values of samples in the sliding window. If the jump of values between samples is not linearly propotional to the expected max_stepsize, then it is labelled a pothole sample.

A pothole sample is one which represents a sudden jump in the values - indicating a noisy tracking of the frequency. The jump in values in a non-noisy signal is expected to be proportional to the distance between the samples.

For instance, if :

>>> a = np.array([10, 2, 6, 10, 12])

If the max step size is 2, then because abs(10-2)>2, it causes a pothole to appear on 2. There is no pothole label on the 2nd index because abs(10-6) is not >4. Because 10 and 6 are two samples apart, the maximum allowed jump in value is max_stepsize*2, which is 4.

For optimal pothole detection the ‘look-ahead’ span of the pothole_inspection_window should at least the size of the longest expected potholes. Smaller window sizes will lead to false negatives.

Parameters

• X (np.array) –

• fs (float>0) –

• max_stepsize (float>0) – The max absolute difference between the values of one sample to the next.

• pothole_inspection_window (float>0, optional) – Defaults to 0.25ms

Returns

pothole_candidates – Boolean array with same size as X. Sample that are True represent pothole candidates.

Return type

np.array

See also

detect_local_potholes()

itsfm.signal_cleaning.onepass_identify_potholes(X, fs, max_stepsize, **kwargs)

itsfm.signal_cleaning.detect_local_potholes(X, max_step_size)

accepts a 1D array and checks the absolute difference between the first sample and all other samples.

The samples with difference greater than the linearly expected increase from max_step_sizes are labelled candidate potholes.

Parameters

• X (np.array) –

• max_step_size (float>=0) –

Returns

candidate_potholes – Boolean array of same size as X

Return type

np.array

itsfm.signal_cleaning.get_all_spikeish_indices(regions)

itsfm.signal_cleaning.find_non_forbidden_index(candidate, forbidden_indices, search_direction, X)

itsfm.signal_cleaning.remove_bursts(X, fs, **kwargs)

Bursts are brief but large jumps in the signal above zero. Even though they satisfy most of the other conditions of beginning above the noise floor and of being above 0 value, they still are too short to be relevant signals.

Parameters

• X (np.array) – The noisy signal to be handled

• fs (float>0) – Sampling rate in Hz.

• min_element_length (float>0, optional) – The minimum length a section of the signal must be to be kept in seconds. Defaults to 5 inter-sample-intervals.

Returns

X_nonspikey – Same size as X, and without very short segments.

Return type

np.array

See also

segments_above_min_duration()

Notes

An inter-sample-interval is defined as 1/fs

itsfm.signal_cleaning.segments_above_min_duration(satisfies_condition, min_samples)

Accepts a boolean array and looks for continuous chunks that are above a minimum length.

Parameters

• satisfies_condition (np.array) – Boolean array where samples with True satisfy a condition.

• min_samples (int >0) – The minimum number of samples a continuous region of True must be to be kept.

Returns

above_min_duration – Same size as satisfies_condition, with only the continuous chunks that are above min_samples.

Return type

np.array

itsfm.signal_cleaning.suppress_background_noise(main_signal, input_audio, **kwargs)

itsfm.signal_cleaning.suppress_frequency_spikes(noisy_profile, input_audio, fs, **kwargs)

itsfm.signal_cleaning.suppress_to_zero(target_signal, basis_signal, threshold, mode='below')

Sets the values of the target signal to zero if the samples in the basis_signal are $geq$ or $leq$ the threshold

Parameters

• basis_signal (target_signal,) –

• threshold (float) –

• mode (['below', 'above'], str) –

Returns

cleaned_signal – A copy of the target signal with the values that are below/above the threshold set to zero

Return type

np.array

Example

# create a basis signal with a ‘weak’ left half and a ‘loud’ right hald # we want to suppress the we >>> basis = np.concatenate((np.arange(10), np.arange(100,200))) >>> target_signal = np.random.normal(0,1,basis.size) >>> cleaned_target = suppress_to_zero(basis, target_signal, 100, mode=’above’)

itsfm.signal_cleaning.clip_tfr(tfr, **kwargs)

Parameters

• tfr (np.array) – 2D array with the time-frequency representation of choice (pwvd, fft etc). The tfr must have real-valued non-negative values as the clip range is defined in dB.

• tfr_cliprange (float >0, optional) – The maximum dynamic range in dB which will be used to track the instantaneous frequency. Defaults to None. See Notes for more details

Returns

clipped_tfr – A 2d array of same shape as tfr, with values clipped between [max, max x 10^(tfr_range/20)]

Return type

np.array

Notes

The tfr_cliprange is used to remove the presence of background noise, faint harmonics or revernberations/echoes in the audio. This of course all assumes that the main signal itself is sufficiently intense in the first place.

After the PWVD time-frequency represenation is made, values below X dB of the maximum value are ‘clipped’ to the same minimum value. eg. if the pwvd had values of [0.1, 0.9, 0.3, 1, 0.001, 0.0006] and the tfr_cliprange is set to 6dB, then the output of the clipping will be [0.5, 0.9, 0.3, 1, 0.5, 0.5]. This step essentially eliminates any variation in the array, thus allowing a clear tracking of the highest component in it.

itsfm.signal_cleaning.conditionally_set_to(X, conditional, bool_state)

Inverts the samples in X where the conditional is True. :param X: Boolean :type X: np.array :param conditional: Boolean :type conditional: np.array :param bool_state: :type bool_state: [True, False]

Returns

cond_set_X – conditionally set X

Return type

np.array

Notes

this function is useful if you want to ‘suppress’ a few samples conditionally based ont he values of the same samples on another array.

Example

>>> x = np.array([True, True, False, False, True])
>>> y = np.array([0,0,10,10,10])
Imagine x is some kind of detection array, while y is the
signal-to-noise ratio at each of the sample points. Of course,
you'd like to discard all the predictions from low SNR measurements.
Let's say you want to keep only those entries in X where y is >1.
>>> x_cond = conditionally_set_to(x, y<10, False)
>>> x_cond

np.array([False, False, False, False, True ])

Batch processing

Runs the batch processing option. The main outputs are the call measurements and the visualisations. (See __main__.py)

$ python -m itsfm -batchfile template_batchfile.csv

Also allows the user to run only one specific row of the whole batch file

$ python -m itsfm -batchfile template_batchfile.csv -one_row 10

The line above loads the 11th row (0-based indexing!!) of the template_batchfile

itsfm.batch_processing.run_from_batchfile(batchfile_path, **kwargs)

Parameters

batchfile_path (str/path) – Path to a batchfile

Keyword Arguments

• one_row (int, optional) – A specific row to be loaded from the whole batchfile The first row starts with 0. Defaults to None

• _from (int, optional) – Row to start the batchfile processing from. Defaults to None

• _till (int, optional) – Row to end the batchfile processing. Defaults to None

itsfm.batch_processing.subset_batch_data(batch_data, **kwargs)

Parameters

batch_data (pd.DataFrame) –

Keyword Arguments

• one_row (int, optional) – Defaults to None

• _from (int, optional) – Defaults to None

• _till (int, optional) – The row number the analysis should run till, including the end point. Remember the row numbering starts from 0! Defaults to None

Returns

subset_batch_data – Either a copy of batch_data or a part of batch_data

Return type

pd.DataFrame

Example

# let’s get only one row from the fake batch data file >>> batch = pd.DataFrame(data={‘a’:range(10), ‘b’:range(10)}) >>> onerow = subset_batch_data(batch, one_row=5) >>> print(onerow) # get a limited range of the dataframe >>> part = subset_batch_data(batch, _from=3, _till=8) >>> print(part)

itsfm.batch_processing.measurement_file_action(**kwargs)

Either lets the measurement file remain, or deletes it if present

Keyword Arguments

del_measurement (boolean) – True means all files starting with ‘measurement’ are deleted

itsfm.batch_processing.onerow_used_properly(**kwargs)

Checks that the -one_row argument is not used in conjunction with -from or -till

itsfm.batch_processing.save_measurements_to_file(output_filepath, audio_file_name, previous_rows, measurements)

Continously saves a row to a csv file and updates it.

Thanks to tmss @ https://stackoverflow.com/a/46775108

Parameters

• output_filepath (str/path) –

• audio_file_name (str.) –

• previous_rows (pd.DataFrame) – All the previous measurements. Can also just have a single row.

• measurements (pd.DataFrame) – Current measurements to be incorporated

Returns

Return type

None, previous rows

Notes

Main side effect is to write an updated version of the output file.

itsfm.batch_processing.load_raw_audio(kwargs)

Takes a dictioanry input. All the parameter names need to be keys in the input dictionary.

Parameters

• audio_path (str/path) – Path to audio file

• channel (int, optional) – Channel number to be loaded - starting from 1! Defaults to 1.

• start,stop (float, optional) –

Returns

raw_audio – The audio corresponding to the start and stop times and the required channel.

Return type

np.array

itsfm.batch_processing.to_separate_from_background(arguments)

itsfm.batch_processing.to_list_w_funcs(X, source_module=<module 'itsfm.measurement_functions' from '/home/docs/checkouts/readthedocs.org/user_builds/itsfm/envs/latest/lib/python3.7/site-packages/itsfm-0.0.1-py3.7.egg/itsfm/measurement_functions.py'>, **kwargs)

Parameters

• X (str) – String defining a list with commas as separators eg. “[func_name1, func_name2] “

• source_module (str, optional) – Defaults to itsfm.measurement_functions

• signs_to_remove (list w str) – Any special signs to remove from each str in the list of comma separated strings. Defaults to None.

Returns

list with functions belonging to the source module

Return type

list_w_funcs

Example

>>> x = "[measure_rms, measure_peak_amplitude]"
>>> list_w_funcs = to_list_w_funcs(x)

itsfm.batch_processing.remove_punctuations(full_str, **kwargs)

Removes spaces, ], and [ in a string. Additional signs can be removed too

Parameters

• full_str (str) – A long string with multiple punctuation marks to be removed (space, comma, ])

• signs_to_remove (list w str', optional) – Additional specific punctuation/s to be removed Defaults to None

Returns

clean_str

Return type

str

itsfm.batch_processing.parse_batchfile_row(one_row)

checks for all user-given arguments and removes any columns with DEFAULT in them.

Parameters

one_row (pd.DataFrame) – A single row with multiple column names, corresponding to compulsory required arguments and the optional ones

Returns

arguments – Simple dictioanry with one entry for each key.

Return type

dictionary

itsfm.batch_processing.make_to_oned_dataframe(oned_series)

Parameters

oned_series (pd.Series) – One dimensional pd.Series with columns and values

Returns

Return type

oned_df

exception itsfm.batch_processing.ImproperArguments