Tweeper 1.4.3 released, twitter to RSS conversion is back once again

Tweeper is a web scraper which converts Twitter and other social media websites to RSS.

On December 27th 2020 version 1.4.3 of Tweeper has been released. The tweeper Debian package has also been updated.

For PHP composer users there is also ao2/tweeper on packagist, which means that tweeper can be installed and run with these commands:

$ composer global require ao2/tweeper
$ ~/.config/composer/vendor/bin/tweeper

Version 1.4.3 is a maintenance release with a couple of fixes.

Here are the NEWS file entries:

News for v1.4.3:
================

  * Fix scraping twitter.com again by impersonating a Google crawler
  * Add check for http response code and return failure for error codes
  * Return failure when instagram.com redirects to login page


CommentiCondividi contenuti

I get this when I try to use

Ritratto di GunnarG

I get this when I try to use compose:

Changed current directory to /home/user/.config/composer
Warning from https://repo.packagist.org: You are using an outdated version of Composer. Composer 2 is now available and you should upgrade. See https://getcomposer.org/2
Using version ^1.4 for ao2/tweeper
./composer.json has been updated
Loading composer repositories with package information
Warning from https://repo.packagist.org: You are using an outdated version of Composer. Composer 2 is now available and you should upgrade. See https://getcomposer.org/2
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

Problem 1
- friendsofphp/php-cs-fixer v2.10.4 requires php ^5.6 || >=7.0 <7.3 -> your PHP version (7.4.3) does not satisfy that requirement.
- friendsofphp/php-cs-fixer v2.10.4 requires php ^5.6 || >=7.0 <7.3 -> your PHP version (7.4.3) does not satisfy that requirement.
- friendsofphp/php-cs-fixer v2.10.4 requires php ^5.6 || >=7.0 <7.3 -> your PHP version (7.4.3) does not satisfy that requirement.
- Installation request for friendsofphp/php-cs-fixer (locked at v2.10.4, required as ^2.10) -> satisfiable by friendsofphp/php-cs-fixer[v2.10.4].

Using composer like this

Ritratto di GunnarG

Using composer like this fixes the issue for me.
composer global require --ignore-platform-reqs ao2/tweeper

I think Instagram broke

Ritratto di Francesco

I think Instagram broke tweeper.

 $ ~/media/vcs/tweeper/./tweeper https://instagram.com/instagram
PHP Warning:  Cannot open instagram page: redirected to Login page.
 in /home/f/media/vcs/tweeper/src/Tweeper.php on line 327

The redirect to the login

Ritratto di ao2

The redirect to the login page is quite normal these days, Instagram detects and blocks scrapers in a lot of ways. Sometimes this redirect can be temporarily worked around by changing the IP address (e.g. by rebooting your modem).

Besides that, scraping Instagram has been indeed broken in Tweeper itself for some time, this change should fix it: https://git.ao2.it/tweeper.git/commitdiff/75c303bf7c570748a5564de6e266cc40f75b1697

Thanks, Antonio

And indeed I git pulled and

Ritratto di Francesco

And indeed I git pulled and it works! tweeper is Godsent, reclaiming the web from social silos. Thanks again!

Since 23/06/2022 at evening

Ritratto di Anonymous

Since 23/06/2022 at evening I'm getting empty list of tweets. For example:

$ tweeper -v0 https://twitter.com/ao2it
{?xml version="1.0"?}
{rss version="2.0" xml:base="https://twitter.com"}
{channel}
{generator}Tweeper{/generator}
{title}Twitter / {/title}
{link}https://twitter.com/ao2it{/link}
{description/}
{/channel}
{/rss}

NOTE: I replaced '<' for '{' and '>' for '}' because the content was unreadable in this web if was written in raw format (with "tag" style).

On 23/06/2002 at the morning it was working but it didn't work at the evening and still isn't working.

I try to purge tweeper package and the php's packages dependencies and I make a clean install from the tweeper 1.4.3 deb package, but stills isn't working (ubuntu). Also try to install on another machine (debian) and also have the same result: no errors, but empty list of tweets.

I'm not sure if tweeper uses twitter API, but checking the changelog seems there isn't any change since June 16th:
https://developer.twitter.com/en/updates/changelog

So, I'm not sure if this is a change of Twitter or the problem is on my machines. Anyone have the same issue? Any workaround to fix this?

Sorry for my english, isn't my mother tongue, and thanks in advance!

PS: Also force my router to

Ritratto di Anonymous

PS: Also force my router to change the IP rebooting it (checked that the IP has changed) and still the same result.

Thanks again!

someone have the same

Ritratto di Anonymous

someone have the same problem?

thanks in advance!

I have the same problem since

Ritratto di mk

I have the same problem since June 23 continuously getting empty contents.

Thanks for reply! hope

Ritratto di Anonymous

Thanks for reply! hope Antonio Ospite (Tweeper's coder) can fix this or explain us a workaround!

Invia nuovo commento

Il contenuto di questo campo è privato e non verrà mostrato pubblicamente. If you have a Gravatar account associated with the e-mail address you provide, it will be used to display your avatar.
  • Indirizzi web o e-mail vengono trasformati in link automaticamente
  • Elementi HTML permessi: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Linee e paragrafi vanno a capo automaticamente.

Ulteriori informazioni sulle opzioni di formattazione

CAPTCHA
Questa domanda serve a verificare che il form non venga inviato da procedure automatizzate
x
c
Z
s
S
L
Enter the code without spaces.