Skip to content

Commit

Permalink
[pulseaudio] Declare pulseaudio module per audio stream (openhab#16254)
Browse files Browse the repository at this point in the history 
* [pulseaudio] Declare pulseaudio module per audio stream

Signed-off-by: Miguel Álvarez <miguelwork92@gmail.com>
  • Loading branch information
@GiviMAD
GiviMAD authored Apr 30, 2024
1 parent 9bcb338 commit 94b4639
Show file tree
Hide file tree
Showing 14 changed files with 707 additions and 505 deletions.
31 changes: 29 additions & 2 deletions bundles/org.openhab.binding.pulseaudio/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,22 +62,49 @@ Sink things can register themselves as audio sink in openHAB. MP3 and WAV files
Use the appropriate parameter in the sink thing to activate this possibility (activateSimpleProtocolSink).
This requires the module **module-simple-protocol-tcp** to be present on the server which runs your openHAB instance. The binding will try to command (if not discovered first) the load of this module on the pulseaudio server.

### Thing Configuration

| Config Name | Item Type | Description |
|-----------------------------|-----------------------------------------------------------------------------------------------------------------|
| name | text | The name of one specific device. You can also use the description |
| activateSimpleProtocolSink | boolean | Activation of a corresponding sink in OpenHAB |
| additionalFilters | text | Additional filters to select the proper device on the pulseaudio server, in case of ambiguity |
| simpleProtocolIdleModules | integer | Number of Simple Protocol TCP Socket modules to keep loaded in the server |
| simpleProtocolMinPort | integer | Min port used by simple protocol module instances created by the binding on the pulseaudio host |
| simpleProtocolMaxPort | integer | Max port used by simple protocol module instances created by the binding on the pulseaudio host |
| simpleProtocolSOTimeout | integer | Socket SO timeout when connecting to pulseaudio server though module-simple-protocol-tcp |

## Audio source

Source things can register themselves as audio source in openHAB.
WAV input format, rate and channels can be configured on the thing configuration. (defaults to pcm_signed,16000,1)
Use the appropriate parameter in the source thing to activate this possibility (activateSimpleProtocolSource).
This requires the module **module-simple-protocol-tcp** to be present on the target pulseaudio server. The binding will load this module on the pulseaudio server.

### Thing Configuration

| Config ID | Item Type | Description |
|------------------------------|-----------------------------------------------------------------------------------------------------------------|
| name | text | The name of one specific device. You can also use the description |
| activateSimpleProtocolSource | boolean | Activation of a corresponding sink in OpenHAB |
| additionalFilters | text | Additional filters to select the proper device on the pulseaudio server, in case of ambiguity |
| simpleProtocolIdleModules | integer | Number of Simple Protocol TCP Socket modules to keep loaded in the server |
| simpleProtocolMinPort | integer | Min port used by simple protocol module instances created by the binding on the pulseaudio host |
| simpleProtocolMaxPort | integer | Max port used by simple protocol module instances created by the binding on the pulseaudio host |
| simpleProtocolSOTimeout | integer | Socket SO timeout when connecting to pulseaudio server though module-simple-protocol-tcp |
| simpleProtocolSourceFormat | text | The audio format to be used by module-simple-protocol-tcp on the pulseaudio server |
| simpleProtocolSourceRate | integer | The audio sample rate to be used by module-simple-protocol-tcp on the pulseaudio server |
| simpleProtocolSourceChannels | integer | The audio channel number to be used by module-simple-protocol-tcp on the pulseaudio server |

## Full Example

### pulseaudio.things

```java
Bridge pulseaudio:bridge:<bridgname> "<Bridge Label>" @ "<Room>" [ host="<ipAddress>", port=4712 ] {
Things:
Thing sink multiroom "Snapcast" @ "Room" [name="alsa_card.pci-0000_00_1f.3", activateSimpleProtocolSink=true, simpleProtocolSinkPort=4711, additionalFilters="analog-stereo###internal"]
Thing source microphone "microphone" @ "Room" [name="alsa_input.pci-0000_00_14.2.analog-stereo"]
Thing sink multiroom "Snapcast" @ "Room" [name="alsa_card.pci-0000_00_1f.3", activateSimpleProtocolSink=true, additionalFilters="analog-stereo###internal"]
Thing source microphone "microphone" @ "Room" [name="alsa_input.pci-0000_00_14.2.analog-stereo", activateSimpleProtocolSource=true]
Thing sink-input openhabTTS "OH-Voice" @ "Room" [name="alsa_output.pci-0000_00_1f.3.hdmi-stereo-extra1"]
Thing source-output remotePulseSink "Other Room Speaker" @ "Other Room" [name="alsa_input.pci-0000_00_14.2.analog-stereo"]
Thing combined-sink hdmiAndAnalog "Zone 1+2" @ "Room" [name="combined"]
Expand Down
153 changes: 68 additions & 85 deletions ...ulseaudio/src/main/java/org/openhab/binding/pulseaudio/internal/ConvertedInputStream.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javazoom.spi.mpeg.sampled.convert.MpegFormatConversionProvider;
import javazoom.spi.mpeg.sampled.file.MpegAudioFileReader;

import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.UnsupportedAudioFileException;
Expand All @@ -28,160 +28,143 @@
import org.eclipse.jdt.annotation.Nullable;
import org.openhab.core.audio.AudioFormat;
import org.openhab.core.audio.AudioStream;
import org.openhab.core.audio.SizeableAudioStream;
import org.openhab.core.audio.UnsupportedAudioFormatException;
import org.openhab.core.audio.utils.AudioWaveUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.tritonus.share.sampled.file.TAudioFileFormat;

/**
* This class convert a stream to the normalized pcm
* format wanted by the pulseaudio sink
* This class convert a stream to the pcm signed
* format supported by the pulseaudio sink
*
* @author Gwendal Roulleau - Initial contribution
* @author Miguel Álvarez Díez - Extend from AudioStream
*/
@NonNullByDefault
public class ConvertedInputStream extends InputStream {
public class ConvertedInputStream extends AudioStream {

private final Logger logger = LoggerFactory.getLogger(ConvertedInputStream.class);

private static final javax.sound.sampled.AudioFormat TARGET_FORMAT = new javax.sound.sampled.AudioFormat(
javax.sound.sampled.AudioFormat.Encoding.PCM_SIGNED, 44100, 16, 2, 4, 44100, false);
private AudioFormat originalAudioFormat;
private final AudioFormat outputAudioFormat;
private final InputStream pcmInnerInputStream;

private final AudioFormat audioFormat;
private AudioInputStream pcmNormalizedInputStream;

private long duration = -1;
private long length = -1;
private static final Set<String> COMPATIBLE_CODEC = Set.of(AudioFormat.CODEC_PCM_ALAW, AudioFormat.CODEC_PCM_ULAW,
AudioFormat.CODEC_PCM_UNSIGNED);

public ConvertedInputStream(AudioStream innerInputStream)
throws UnsupportedAudioFormatException, UnsupportedAudioFileException, IOException {
this.audioFormat = innerInputStream.getFormat();
this.originalAudioFormat = innerInputStream.getFormat();

String container = originalAudioFormat.getContainer();
if (container == null) {
throw new UnsupportedAudioFormatException("Unknown format, cannot process", innerInputStream.getFormat());
}

if (innerInputStream instanceof SizeableAudioStream sizeableAudioStream) {
length = sizeableAudioStream.length();
if (container.equals(AudioFormat.CONTAINER_WAVE)) {
if (originalAudioFormat.getFrequency() == null || originalAudioFormat.getChannels() == null
|| originalAudioFormat.getBitRate() == null || originalAudioFormat.getCodec() == null
|| originalAudioFormat.getBitDepth() == null || originalAudioFormat.isBigEndian() == null) {
// parse it by ourself to maybe get missing information :
this.originalAudioFormat = AudioWaveUtils.parseWavFormat(innerInputStream);
}
}

pcmNormalizedInputStream = getPCMStreamNormalized(getPCMStream(new BufferedInputStream(innerInputStream)));
if (AudioFormat.CODEC_PCM_SIGNED.equals(originalAudioFormat.getCodec())) {
outputAudioFormat = originalAudioFormat;
pcmInnerInputStream = innerInputStream;
if (container.equals(AudioFormat.CONTAINER_WAVE)) {
AudioWaveUtils.removeFMT(innerInputStream);
}

} else {
pcmInnerInputStream = getPCMStream(new BufferedInputStream(innerInputStream));
var javaAudioFormat = ((AudioInputStream) pcmInnerInputStream).getFormat();
int bitRate = (int) javaAudioFormat.getSampleRate() * javaAudioFormat.getSampleSizeInBits()
* javaAudioFormat.getChannels();
outputAudioFormat = new AudioFormat(AudioFormat.CONTAINER_NONE, AudioFormat.CODEC_PCM_SIGNED,
javaAudioFormat.isBigEndian(), javaAudioFormat.getSampleSizeInBits(), bitRate,
(long) javaAudioFormat.getSampleRate(), javaAudioFormat.getChannels());
}
}

@Override
public int read(byte @Nullable [] b) throws IOException {
return pcmNormalizedInputStream.read(b);
return pcmInnerInputStream.read(b);
}

@Override
public int read(byte @Nullable [] b, int off, int len) throws IOException {
return pcmNormalizedInputStream.read(b, off, len);
return pcmInnerInputStream.read(b, off, len);
}

@Override
public byte[] readAllBytes() throws IOException {
return pcmNormalizedInputStream.readAllBytes();
return pcmInnerInputStream.readAllBytes();
}

@Override
public byte[] readNBytes(int len) throws IOException {
return pcmNormalizedInputStream.readNBytes(len);
return pcmInnerInputStream.readNBytes(len);
}

@Override
public int readNBytes(byte @Nullable [] b, int off, int len) throws IOException {
return pcmNormalizedInputStream.readNBytes(b, off, len);
return pcmInnerInputStream.readNBytes(b, off, len);
}

@Override
public int read() throws IOException {
return pcmNormalizedInputStream.read();
return pcmInnerInputStream.read();
}

@Override
public void close() throws IOException {
pcmNormalizedInputStream.close();
}

/**
* Ensure right PCM format by converting if needed (sample rate, channel)
*
* @param pcmInputStream
*
* @return A PCM normalized stream (2 channel, 44100hz, 16 bit signed)
*/
private AudioInputStream getPCMStreamNormalized(AudioInputStream pcmInputStream) {
javax.sound.sampled.AudioFormat format = pcmInputStream.getFormat();
if (format.getChannels() != 2
|| !format.getEncoding().equals(javax.sound.sampled.AudioFormat.Encoding.PCM_SIGNED)
|| Math.abs(format.getFrameRate() - 44100) > 1000) {
logger.debug("Sound is not in the target format. Trying to reencode it");
return AudioSystem.getAudioInputStream(TARGET_FORMAT, pcmInputStream);
} else {
return pcmInputStream;
}
}

public long getDuration() {
return duration;
pcmInnerInputStream.close();
}

/**
* If necessary, this method convert MP3 to PCM, and try to
* extract duration information.
* If necessary, this method convert to target PCM
*
* @param resetableInnerInputStream A stream supporting reset operation
* (reset is mandatory to parse formation without loosing data)
*
* @return PCM stream
* @throws UnsupportedAudioFileException
* @throws IOException
* @throws UnsupportedAudioFormatException
* @throws IOException
*/
private AudioInputStream getPCMStream(InputStream resetableInnerInputStream)
throws UnsupportedAudioFileException, IOException, UnsupportedAudioFormatException {
if (AudioFormat.MP3.isCompatible(audioFormat)) {
MpegAudioFileReader mpegAudioFileReader = new MpegAudioFileReader();

if (length > 0) { // compute duration if possible
AudioFileFormat audioFileFormat = mpegAudioFileReader.getAudioFileFormat(resetableInnerInputStream);
if (audioFileFormat instanceof TAudioFileFormat) {
Map<String, Object> taudioFileFormatProperties = ((TAudioFileFormat) audioFileFormat).properties();
if (taudioFileFormatProperties.containsKey("mp3.framesize.bytes")
&& taudioFileFormatProperties.containsKey("mp3.framerate.fps")) {
Integer frameSize = (Integer) taudioFileFormatProperties.get("mp3.framesize.bytes");
Float frameRate = (Float) taudioFileFormatProperties.get("mp3.framerate.fps");
if (frameSize != null && frameRate != null) {
duration = Math.round((length / (frameSize * frameRate)) * 1000);
logger.debug("Duration of input stream : {}", duration);
}
}
}
resetableInnerInputStream.reset();
}

if (AudioFormat.CODEC_MP3.equals(originalAudioFormat.getCodec())) {
logger.debug("Sound is a MP3. Trying to reencode it");
// convert MP3 to PCM :
AudioInputStream sourceAIS = mpegAudioFileReader.getAudioInputStream(resetableInnerInputStream);
AudioInputStream sourceAIS = new MpegAudioFileReader().getAudioInputStream(resetableInnerInputStream);
javax.sound.sampled.AudioFormat sourceFormat = sourceAIS.getFormat();

MpegFormatConversionProvider mpegconverter = new MpegFormatConversionProvider();
int bitDepth = sourceFormat.getSampleSizeInBits() != -1 ? sourceFormat.getSampleSizeInBits() : 16;
javax.sound.sampled.AudioFormat convertFormat = new javax.sound.sampled.AudioFormat(
javax.sound.sampled.AudioFormat.Encoding.PCM_SIGNED, sourceFormat.getSampleRate(), 16,
sourceFormat.getChannels(), sourceFormat.getChannels() * 2, sourceFormat.getSampleRate(), false);
javax.sound.sampled.AudioFormat.Encoding.PCM_SIGNED, sourceFormat.getSampleRate(), bitDepth,
sourceFormat.getChannels(), 2 * sourceFormat.getChannels(), sourceFormat.getSampleRate(), false);

return mpegconverter.getAudioInputStream(convertFormat, sourceAIS);
} else if (AudioFormat.WAV.isCompatible(audioFormat)) {
// return the same input stream, but try to compute the duration first
AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(resetableInnerInputStream);
if (length > 0) {
int frameSize = audioInputStream.getFormat().getFrameSize();
float frameRate = audioInputStream.getFormat().getFrameRate();
float durationInSeconds = (length / (frameSize * frameRate));
duration = Math.round(durationInSeconds * 1000);
logger.debug("Duration of input stream : {}", duration);
}
} else if (COMPATIBLE_CODEC.contains(originalAudioFormat.getCodec())) {
long frequency = Optional.ofNullable(originalAudioFormat.getFrequency()).orElse(44100L);
int channel = Optional.ofNullable(originalAudioFormat.getChannels()).orElse(1);
javax.sound.sampled.AudioFormat targetFormat = new javax.sound.sampled.AudioFormat(frequency, 16, channel,
true, false);
AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(targetFormat,
AudioSystem.getAudioInputStream(resetableInnerInputStream));
return audioInputStream;
} else {
throw new UnsupportedAudioFormatException("Pulseaudio audio sink can only play pcm or mp3 stream",
audioFormat);
originalAudioFormat);
}
}

@Override
public AudioFormat getFormat() {
return outputAudioFormat;
}
}
Loading

0 comments on commit 94b4639

Please sign in to comment.