Tweeper 0.4 released, scraping Facebook and Instagram

On September 13th 2015 I released version 0.4 of tweeper, a web scraper which converts Twitter and other sites to RSS.

A snippet from the NEWS file:

News for v0.4:
==============

  * Make the generated RSS validate with feedvalidator.org
  * Fix support for Dilbert.com
  * Add support for Instragram.com
  * Add support for public pages on Facebook.com
  * Make tweeper work with the PHP built-in web server
  * Misc fixes to code and documentation

Adding support for Instagram.com was interesting: the site serves the content as JSON and to reuse the generic XSLT approach I decided to convert the JSON to XML using XML_Serializer. I read that there are proposals for direct JSON transformation in XSLT 3.0 but I am not sure if the functionality is available anywhere yet.

Support for public pages on Facebook.com was also added, because since June 23rd Facebook dropped the page RSS Feed endpoint; it was quite easy: they just “hide” the relevant content in HTML comments.

Now I would like to get in touch with some developers more into PHP than I am, to help me cleanup the code, make it a library and maybe upload it to packagist to make it available to composer users.

Anyone?


CommentsSyndicate content

Hi Antonio, Thanks for

Anonymous's picture

Hi Antonio,

Thanks for continuing to work on this great program, not everyone wants a twitter account but may want to follow.

However, it would be really nice if this worked with thunderbird - am i missing something obvious? Any plans to have thunderbird support?

Thanks again!

See if Thunderbird can use

ao2's picture

See if Thunderbird can use commands/filters as RSS feed sources. I don't know.

If not you can always use tweeper with a local web server (even the php builtin webserver) as explained in the tweeper man page, and set the local feed URL in Thunderbird.

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account associated with the e-mail address you provide, it will be used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
c
S
z
1
5
v
Enter the code without spaces.