HARK FORUM › Recording audio stream from ROS – data type error
August 12, 2019 at 4:50 pm #1105
I am new to Hark and I am trying to record the audio captured with a HSR robot’s built-in head microphone. The following are the parameters of the stream generated by the robot:
PARAMETERS * /audio/audio_capture/bitrate: 128 * /audio/audio_capture/channels: 1 * /audio/audio_capture/device: * /audio/audio_capture/format: mp3 * /audio/audio_capture/sample_rate: 16000 * /rosdistro: kinetic * /rosversion: 1.12.14 NODES /audio/ audio_capture (audio_capture/audio_capture)
I am trying to record the streamed audio with the AudioStreamFromRos node (Attached you find a screenshot of my Hark network). However, when I execute the network I get the following error message:
[ERROR] [1565594592.746777286]: Client [/MY_HARK_MASTER_NODE] wants topic /audio/audio to have datatype/md5sum [hark_msgs/HarkWave/24c5654436a3ff03c563377fdbcc56a1], but our version has [audio_common_msgs/AudioData/f43a8e1b362b75baa741461b46adc7e0]. Dropping connection.
How can I configure the node to make the data type compatible?
Attachments:August 13, 2019 at 11:31 am #1107lapus.erParticipant
Can you please attach the actual network file that you used.
EarlAugust 13, 2019 at 9:14 pm #1109
here is the network.August 13, 2019 at 9:24 pm #1110August 16, 2019 at 6:05 pm #1122Masayuki TakigahiraModerator
You need to use
hark_msgs/HarkWavein your workspace.
In your case,
audio_common_msgs/AudioDataseems to store
uint8array, so you will first need to expand it to raw PCM data.
Second, the data structure of
hark_msgs/HarkWaveis as follows.
user@ubuntu:~$ rosmsg show hark_msgs/HarkWave std_msgs/Header header uint32 seq time stamp string frame_id int32 count int32 nch int32 length int32 data_bytes hark_msgs/HarkWaveVal src float32 wavedata
wavedatais a raw PCM data array. Since HARK is not aware of the number of bits, it simply casts an integer value to a floating point type. In other words,
data_bytesis the data size. In other words, it is the size of
float(4 bytes) multiplied by the size of
lengthis the number of samples per frame handled by HARK. The initial value of HARK is
512. Since HARK processed frame by frame, in other words, the size of
nchis the number of channels. Your device seems to be 1ch, so it should be
1. For microphone array data, a larger number will be stored.
countis the frame count. Since HARK processing frame by frame, it is necessary to know what frame the data is. In other words, it is incremented as the frame advances. The first frame number is
There is a final note. In order to prevent problems in FFT/IFFT processing, etc., the frames processing by HARK are subject to sample overlap processing.
The following image may help you understand.
m.takigahiraAugust 23, 2019 at 12:28 am #1123
I am using the package
ros-kinetic-audio-captureto stream the audio from the built-in microphone of the robot.
With this package it is possible to stream audio data in wave format:
PARAMETERS * /audio_capture/channels: 1 * /audio_capture/depth: 16 * /audio_capture/device: plughw:1,0 * /audio_capture/format: wave * /audio_capture/sample_rate: 16000 * /rosdistro: kinetic * /rosversion: 1.12.14 NODES / audio_capture (audio_capture/audio_capture)
The data structure of the data stream
audio_common_msgs/AudioDatais as follows:
user@laptop:~$ rosmsg show audio_common_msgs/AudioData uint8 data
Therefore, it is not compatible with the data structure of
Is there a practical way to make the data structure of
audio_common_msgs/AudioDatacompatible with Hark?
Otherwise, would you suggest another package different than
ros-kinetic-audio-capturein order to have Hark-compatible access to the data from the robot’s built-in microphone?
Thanks in advance.August 26, 2019 at 6:47 pm #1136nterakadoModerator
Please see previous answer on how to convert Structure.
For frame proccessing, refer to the
The source code can be obtained with the following command on Ubuntu.
apt source hark-core
HARK SupportAugust 28, 2019 at 12:55 am #1143Thomas TannousParticipant
I’m also working on this currently.
audio_capturepublish wave format. I tried to write a converter
AudioData (uint8 wave)to
HarkWavefollowing the instructions mentioned in this answer.
Here is what I ended up with:
dhw = HarkWave() dhw.header.stamp = rospy.Time.now() dhw.header.frame_id = str(self.count) dhw.count = self.count harkwaveval = HarkWaveVal(wavedata=data_uint8) result =  result.append(harkwaveval) dhw.src = result dhw.nch = 1 # one channel dhw.length = 320 dhw.data_bytes = 320 * 4 # float(4bytes) times length of data
When I run my network (attached below) it’s only receiving one message from ros and afterwards the network stops with the message “Normally finished”.
So my question is: Why does it stop, since there is no error regarding the network and the topic is still actively pushing messages?
Thanks in advance.
Attachments:August 30, 2019 at 12:10 pm #1152nterakadoModerator
Is the sheet with
SaveWavePCMset to iterator?
If it is a subnet, it will only be executed once.
HARK support team.
September 2, 2019 at 10:10 am #1157tank1199Participant
- This reply was modified 1 year, 11 months ago by nterakado.
> When I run my network (attached below) it’s only receiving one message from ros and afterwards the network stops with the message “Normally finished”.
When creating a new sheet using HarkDesigner, by default it is set to subnet which runs the network once. Please change it to iterator so that the network runs in a loop.
If it is not too much trouble, would you mind uploading a sample of the audio you are trying to stream as well as the network file?
I noticed that you did not have a condition, are you getting any errors when running the network?
- You must be logged in to reply to this topic.