StreamFinder - Fetch actual raw streamable URLs from various radio-station, video & podcast websites.
To install this module, run the following commands: perl Makefile.PL make make test make install
This module is Copyright (C) 2017-2024 by
Jim Turner, <turnerjw784 at yahoo.com>
<turnerjw784 at yahoo.com>
Email: turnerjw784@yahoo.com
All rights reserved.
You may distribute this module under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl README file.
#!/usr/bin/perl use strict; use StreamFinder; die "..usage: $0 URL\n" unless ($ARGV[0]); my $station = new StreamFinder($ARGV[0]); die "Invalid URL or no streams found!\n" unless ($station); my $firstStream = $station->get(); print "First Stream URL=$firstStream\n"; my $url = $station->getURL(); print "Stream URL=$url\n"; my $stationTitle = $station->getTitle(); print "Title=$stationTitle\n"; my $stationDescription = $station->getTitle('desc'); print "Description=$stationDescription\n"; my $stationID = $station->getID(); print "Station ID=$stationID\n"; my $artist = $station->{'artist'}; print "Artist=$artist\n" if ($artist); my $genre = $station->{'genre'}; print "Genre=$genre\n" if ($genre); my $icon_url = $station->getIconURL(); if ($icon_url) { #SAVE THE ICON TO A TEMP. FILE: print "Icon URL=$icon_url=\n"; my ($image_ext, $icon_image) = $station->getIconData(); if ($icon_image && open IMGOUT, ">/tmp/${stationID}.$image_ext") { binmode IMGOUT; print IMGOUT $icon_image; close IMGOUT; print "...Icon image downloaded to (/tmp/${stationID}.$image_ext)\n"; } } my $stream_count = $station->count(); print "--Stream count=$stream_count=\n"; my @streams = $station->get(); foreach my $s (@streams) { print "------ stream URL=$s=\n"; }
StreamFinder accepts a webpage URL for a valid radio station, video, or podcast / episode URL on supported websites and returns the actual stream URL(s), title, and cover art icon for that station / podcast / video. The purpose is that one needs one of these URLs in order to have the option to stream the station / podcast / video in one's own choice of media player software rather than using their web browser and accepting flash, ads, javascript, cookies, trackers, web-bugs, and other crapware associated with that method of play. The author created and uses his own custom all-purpose media player called "Fauxdacious Media Player" (his custom forked version of the open-source "Audacious Audio Player). "Fauxdacious" (https://wildstar84.wordpress.com/fauxdacious/) incorporates this module via a Perl helper-script to decode and play streams, along with their titles / station names, and station / podcast / video icons, artists / channel names, genres, and descriptions!
Please NOTE: StreamFinder is a module, NOT a standalone application. It is designed to be used by other Perl applications. To create your own very simple application just to fetch stream data manually, simply grab the code in the SYNOPSIS section above, save it to an executable text file, ie. StreamFinder.pl, and run it from the command line with a supported streaming site URL as the argument. You can then edit it to tailor it to your needs.
The currently-supported websites are: podcasts.apple.com podcasts (StreamFinder::Apple), bitchute.com videos (StreamFinder::Bitchute), blogger.com videos (StreamFinder::Blogger), ugetube.com videos (StreamFinder::BrandNewTube), brighteon.com videos (StreamFinder::Brighteon), castbox.fm podcasts (StreamFinder::Castbox), podcasts.google.com podcasts (StreamFinder::Google), iheart.com (aka iheartradio.com) radio stations and podcasts (StreamFinder::IHeartRadio), www.internet-radio.com radio stations (StreamFinder::InternetRadio), onlineradiobox.com radio stations (StreamFinder::OnlineRadiobox), odysee.com videos (StreamFinder::Odysee), podbean.com podcasts (StreamFinder::Podbean), podcastaddict.com podcasts (StreamFinder::PodcastAddict) (DEPRECIATED), podchaser.com podcasts (StreamFinder::Podchaser), radio.net radio stations (StreamFinder::RadioNet), rcast.net radio stations (StreamFinder::Rcast), rumble.com videos (StreamFinder::Rumble), sermonaudio.com sermons: audio and video (StreamFinder::SermonAudio), soundcloud.com (non-paywalled) songs (StreamFinder::SoundCloud) (DEPRECIATED), spreaker.com podcasts (StreamFinder::Spreaker), subsplash.com podcasts (StreamFinder::Subsplash) (EXPERIMENTAL), tunein.com (non-paywalled) radio stations and podcasts (StreamFinder::Tunein), vimeo.com videos (StreamFinder::Vimeo), youtube.com, et. al and other sites that youtube-dl supports (StreamFinder::Youtube), zeno.fm radio stations and podcasts (StreamFinder::Zeno), and StreamFinder::Anystream - search any (other) webpage URL (not supported by any of the other submodules) for streams.
NOTE: StreamFinder::Podcastaddict is now considered depreciated and may be removed in a later StreamFinder release as it now requires a specific valid episode page to fetch streams from, as Podcastaddict.com has javascripted up their podcast pages now to the point that it is no longer possible to obtain a playlist or first episode from them via our scripts.
NOTE: StreamFinder::LinkTV has been removed as that site no longer provides streams anymore but only links to the various (and diverse) streaming sites that provide their own streams. Some may possibly work via StreamFinder::Youtube or StreamFinder::AnyStream.
NOTE: StreamFinder::Goodpods has been removed, as that site has redone itself in javascript as to no longer be scrapable for streams.
NOTE: Users should also consider StreamFinder::SoundCloud to now be depreciated, as they've added cookie and tracker requirements making it impossible to search for songs on their site without enabling, but song URLs (when known) seem to still work for now, but without channel/artist icons. (Privacy-minded individuals should now be cautious while using this site).
NOTE: For many sites, ie. Youtube, Vimeo, Apple, Spreaker, Castbox, Google, etc. the "station" object actually refers to a specific video or podcast episode, but functions the same way.
Each site is supported by a separate subpackage (StreamFinder::Package), which is determined and selected based on the URL argument passed to it when the StreamFinder object is created. The methods are overloaded by the selected subpackage's methods. An example would be StreamFinder::Youtube.
Please see the POD. documentation for each subpackage for important additional information on options and features specific to each site / subpackage!
One or more playable streams can be returned for each station / video / podcast, along with at least a "title" (station name / video or podcast episode title) and an icon image URL ("iconurl" - if found). Additional information that MAY be fetched is a (larger?) banner image ("imageurl"), a (longer?) "description", an "artist" / author, a "genre", and / or a "year" (podcasts, videos, etc.), an AlbumArtist / channel URL, and possibly a second icon image for the channel (podcasts and videos). Some sites also provide radio stations' FCC call letters ("fccid"). For icon and image URLs, functions exist (getIconData() and getImageData()) to fetch the actual binary data and mime type for downloading to local storage for use by your application or preferred media player. NOTE: StreamFinder::Anystream is not able to return much beyond the stream URLs it finds, but please see it's POD documentation for details on what it is able to return.
If you have another streaming site that is not supported, first, make sure you have youtube-dl installed and see if StreamFinder::Youtube can successfully fetch any streams for it. If not, then please file a feature request via email or the CPAN bug system, or (for faster service), provide a Perl patch module / program source that can extract some or all of the necessary information for streams on that site and I'll consider it! The easiest way to do this is to take one of the existing submodules, copy it to "StreamFinder::YOURSITE.pm", modify it (and the POD docs) to your specific site's needs, test it on several of their pages (see the "SYNOPSIS" code above), and send it to me (That's what I do when I want to add a new site)!
Accepts a URL and creates and returns a new station, video, or podcast object, or undef if the URL is not a valid station or no streams are found.
NOTE: Depending on the type of site being queried, the "station object" can be either a streaming station, a video, or a podcast, but works the same way (method calls, arguments, etc.).
NOTE: A full URL must be specified here, but if using any of the subpackage modules directly instead, then either a full URL OR just the station / video / podcast's site ID may be used! Reason being that this function parses the full URL to determine which subpackage (site) module to use.
options can vary depending on the type of site that is being queried. One option common to all sites is -debug, which turns on debugging output. A numeric option can follow specifying the level (0, 1, or 2). 0 is none, 1 is basic, 2 is detailed. Default: 1 (if -debug is specified). Warning: 2 will dump a ton of output (mostly the HTML of the web page being parsed!
One specific option (-omit, added as of v1.45) permits omitting specific submodules which are currently installed from being considered. For example, to NOT handle Youtube videos nor use the fallback "Anystream" module, specify: -omit => "Youtube,Anystream", which will cause StreamFinder::Anystream and StreamFinder::Youtube to not be used for the stream search. Default is for all installed submodules to be considered. NOTE: Omitting a module from being considered when seeking to match the correct module by site URL does NOT prevent that module from being invoked by a selected module for an embedded link, OR in the case of StreamFinder::Youtube being omitted, will still be invoked, if required or needed by a non-omitted module initially selected!
Another global option (applicable to all submodules) is the -secure option who's argument can be either 0 or 1 (false or true). If 1, then only secure ("https://") streams will be returned. NOTE, it's possible that some sites may only contain insecure ("http://") streams, which won't return any streams if this option is specified. Therefore, it may be necessary, if setting this option globally, to set it to zero in the config. files for those specific modules, if you determine that to be the case (I have not tested all sites for that). Default: -secure is 0 (false) - return all streams (http and https).
Any other options (including -debug) will be passed to the submodule (if any) that handles the URL you pass in, but note, submodules accept different options and ignore ones they do not recognize. Valid values for some options can also vary across different submodules. A better way to change default options for one or more submodules is to set up submodule configuration files for the ones you wish to change.
Additional options:
-hls_bandwidth => "number"
Limit HLS (m3u8) streams that contain a list of other HLS streams of varying BANDWIDTH values (in BITS per second) by selecting the highest bitrate stream at or below the specified limit when $stream->getURL() is called.
DEFAULT -none- (no limiting by bitrate).
-log => "logfile"
Specify path to a log file. If a valid and writable file is specified, A line will be appended to this file every time one or more streams is successfully fetched for a url.
DEFAULT -none- (no logging).
-logfmt specifies a format string for lines written to the log file.
DEFAULT "[time] [url] - [site]: [title] ([total])".
The valid field [variables] are: [stream]: The url of the first/best stream found. [site]: The site (submodule) name matching the webpage url. [url]: The url searched for streams. [time]: Perl timestamp when the line was logged. [title], [artist], [album], [description], [year], [genre], [total], [albumartist]: The corresponding field data returned (or "-na-", if no value).
Returns an array of strings representing all stream URLs found. If "playlist" is specified, then an extended m3u playlist is returned instead of stream url(s). NOTE: For podcast sites, if an author / channel page url is given, rather than an individual podcast episode's url, get() returns the first (latest?) podcast episode found, and get("playlist") returns an extended m3u playlist containing the urls, titles, etc. for all the podcast episodes found on that page url from latest to oldest.
Similar to get() except it only returns a single stream representing the first valid stream found.
Current options are: "random", "nopls", and "noplaylists". By default, the first ("best"?) stream is returned. If "random" is specified, then a random one is selected from the list of streams found. If "nopls" is specified, and the stream to be returned is a ".pls" playlist, it is first fetched and the first entry (or a random entry if "random" is specified) is returned. This is needed by Fauxdacious Mediaplayer. If "noplaylists" is specified, and the stream to be returned is a "playlist" (either .pls or .m3u? extension), it is first fetched and the first entry (or a random entry if "random" is specified) in the playlist is returned.
Returns the number of streams found for the station.
Returns the station's site ID (default), or station's FCC call-letters ("fccid") for applicable sites and stations.
Returns the station's title, (or long description, if "desc" specified).
NOTE: Some sights do not support a separate long description field, so if none found, the standard title field will always be returned.
Returns the URL for the station's "cover art" icon image, if any.
Some video and podcast sites will also provide a separate artist/channel icon. If 'artist' is specified, this icon url is returned instead, if any.
Returns a two-element array consisting of the extension (ie. "png", "gif", "jpeg", etc.) and the actual icon image (binary data), if any. This makes it easy to download the image to local storage for use by your preferred media player.
Some video and podcast sites will also provide a separate artist/channel icon. If 'artist' is specified, this icon's data is returned instead, if any.
Returns the URL for the station's "cover art" banner image, if any.
NOTE: If no "banner image" (usually a larger image) is found, the "icon image" URL will be returned.
Some video and podcast sites will also provide a separate artist/channel image (usually larger). If 'artist' is specified, this icon url is returned instead, if any.
Returns a two-element array consisting of the extension (ie. "png", "gif", "jpeg", etc.) and the actual station's banner image (binary data). This makes it easy to download the image to local storage for use by your preferred media player.
NOTE: If no "banner image" (usually a larger image) is found, the "icon image" data, if any, will be returned.
Some video and podcast sites will also provide a separate artist/channel image (usually larger). If 'artist' is specified, this icon's data is returned instead, if any.
Returns the station / podcast / video's type (submodule-name). (one of: "Anystream", "Apple", "BitChute", "Blogger", "Youtube", etc. - depending on the sight that matched the URL).
The default root location directory for StreamFinder configuration files is "~/.config/StreamFinder". To use an alternate location directory, specify it in the "STREAMFINDER" environment variable, ie.: $ENV{STREAMFINDER} = "/etc/StreamFinder".
Optional text file for specifying various configuration options. Each option is specified on a separate line in the formats below: NOTE: Do not follow the lines with a semicolon, comma, or any other separator. Non-numeric values should be surrounded with quotes, either single or double. Blank lines and lines beginning with a "#" sign as their first non-blank character are ignored as comments.
'option' => 'value' [, ...]
'option' => ['value1', 'value2', ...] [, ...]
'option' => {'key1' => 'value1', 'key2' => 'value2', ...} [, ...]
and the options are loaded into a hash used by all sites (submodules) that support them. Valid options include -debug => [0|1|2] and most of the LWP::UserAgent options.
Optional text file for specifying various configuration options for a specific site (submodule, ie. "Youtube" for StreamFinder::Youtube). Each option is specified on a separate line in the formats below:
and the options are loaded into a hash used only by the specific (submodule) specified. Valid options include -debug => [0|1|2] and most of the LWP::UserAgent options.
NOTE: Options specified here override any specified in ~/.config/StreamFinder/config.
NOTE: Options specified in the options parameter list of the new() function will override those corresponding options specified in these files.
URI::Escape, HTML::Entities, LWP::UserAgent
youtube-dl, or other compatable program such as yt-dlp, etc. (for Youtube, Bitchute, Blogger, Brighteon, Odysee, Vimeo) NOTE: Required for Youtube, Odysee, and SoundCloud to work.
wget
Please report any bugs or feature requests to bug-streamFinder at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=StreamFinder. I will be notified, and then you'llautomatically be notified of progress on your bug as I make changes.
bug-streamFinder at rt.cpan.org
You can find documentation for this module with the perldoc command.
perldoc StreamFinder
You can also look for information at:
Fauxdacious media player - (https://wildstar84.wordpress.com/fauxdacious)
RT: CPAN's request tracker (report bugs here)
http://rt.cpan.org/NoAuth/Bugs.html?Dist=StreamFinder
CPAN Ratings
http://cpanratings.perl.org/d/StreamFinder
Search CPAN
http://search.cpan.org/dist/StreamFinder/
Copyright 2017-2024 Jim Turner.
This program is free software; you can redistribute it and/or modify it under the terms of the the Artistic License (2.0). You may obtain a copy of the full license at:
http://www.perlfoundation.org/artistic_license_2_0
Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license.
If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license.
This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder.
This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed.
Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
To install StreamFinder, copy and paste the appropriate command in to your terminal.
cpanm
cpanm StreamFinder
CPAN shell
perl -MCPAN -e shell install StreamFinder
For more information on module installation, please visit the detailed CPAN module installation guide.