Advertisement:

Author Topic: Automatic Twitter Embedding - Code provided  (Read 11618 times)

Offline dimspace

  • Jr. Member
  • **
  • Posts: 343
    • @dimspace on Twitter
    • Velorooms.com
Automatic Twitter Embedding - Code provided
« on: October 31, 2014, 01:11:55 PM »
Hokay.. Some basic background on this.

We previously used a service from embed tweet to auto embed tweets within our forum but due to various twitter restrictions notably the number of calls this stopped working so we figured out our own solution.

Weve got to the point where we now have working, automated twitter embedding on our site velorooms, but the code is very much a "hack" and requires polishing, and packaging for use by the community.

Firstly, some background on how we did it and why..

Quote
I completed the Twitter embed fix today.

Operating in more or less the same way as the previous implementation, it runs out of two files and a new table in the VR database. The main difference is that the magic happens with VR rather than a third-party service.

Rationale

A quick restatement of why this was necessary: Twitter is happy to supply you the content of a tweet on request but to prevent oversubscription by hacking or lazy site owners, we're limited to 180 requests per fifteen minutes.

On a forum, that limit could be reached pretty quickly with multiple users accessing multiple embedded tweets per page, so we want to cache every request in order not to have to go back to Twitter every single time a tweet is viewed on a page.

JS: http://velorooms.com/vr-js/embed_v2.js

This is the same JS file as for the former EmbedTweet.com solution. I commented out the former code and implemented a bootstrap with jQuery, which is already in general use on the VR website.

The principle for this part is the same as before: scan the current page for tweets in the form of links and grab the tweet ID. For each ID found on the page, we make an AJAX call to a PHP page on the VR server (it would have been good to put all IDs into a single request but Twitter itself won't allow multiple calls, so it wasn't worth it).

PHP / MySQL / JSON tweet loop: http://velorooms.com/tweet-cache.php?id=[Tweet ID]

The PHP page receives the request with the tweet ID attached as a querystring and it checks the new MySQL table tweet_cache for the existence of that tweet ID.

If it isn't already found there, then we know that we have to ask Twitter for the tweet. This comes back as JSON and we store it in the database ready for the next time it's requested. We then pass the result - again as JSON - as a response to the original JS AJAX request and the JS plops it onto the page right where the tweet link was. If the tweet ID was already in the DB, then we pass that back as a reponse instead of bothering Twitter for it

Secondly, the parts involved in making this work.

Part 1 - JS file called Embed_v2 which is in the scripts directory.

Code: [Select]
$(document).ready(function() {

var oTwitter = $('a[href*="twitter.com"][href*="/status"]');
if (oTwitter.length > 0) {
oTwitter.each(function() {
var oHolder = $(this);
var sStr = $(this).attr('href');
sStr = sStr.replace(/\/+$/, "");
sStr = sStr.substr(sStr.lastIndexOf('/') + 1);
$.getJSON( "/tweet-cache.php?id=" + sStr, function(data) {
oHolder.before(data.html);
oHolder.text('Source: ' + oHolder.text()).wrap('<div class="quoteheader"></div>');
});
});
//$.getScript('https://platform.twitter.com/widgets.js');
}

});

part 2 - tweet-cache.php which is placed in the root directory

Code: [Select]
<?php

/*

Rough take on Twitter oEmbed for VeloRooms
by L'arri : voici.l.arriviste@gmail.com
27 October 2014

*/

// I used SMF items where possible for database interactions and required includes.

define('SMF'1);
ob_start();

foreach (array(
'db_character_set''cachedir') as $variable)
if (isset($GLOBALS[$variable]))
unset($GLOBALS[$variable], $GLOBALS[$variable]);

require_once(
dirname(__FILE__) . '/Settings.php');

// Make absolutely sure the cache directory is defined.
if ((empty($cachedir) || !file_exists($cachedir)) && file_exists($boarddir '/cache'))
$cachedir $boarddir '/cache';

require_once(
$sourcedir '/Load.php');

// Handle SMF maintenance
if (!empty($maintenance) && $maintenance == 2) {
//db_fatal_error();
echo '{"html":"Tweet error (database unavailable)"}';
}

// Create a variable to store some SMF specific functions in.
$smcFunc = array();

// This is where SMF's database connection is initiated...
loadDatabase();

global 
$smcFunc;

function 
check_cache($tweet,$smcFunc){

$request $smcFunc['db_query']('','SELECT html from tweet_cache where tweetid ='.$tweet, array());
if (!$request) {
//db_fatal_error();
echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (database unavailable)</p>"}';
}
if ($smcFunc['db_num_rows']($request) == 0) {
add_cache($tweet,$smcFunc);
}
else {
while ($row $smcFunc['db_fetch_row']($request))
echo '{"html" : "' $row[0] .'"}';
}
$smcFunc['db_free_result']($request);

}

function 
add_cache($tweet,$smcFunc){

$twitterapi_url "https://api.twitter.com/1/statuses/oembed.json?id=";
$twitterapi_url $twitterapi_url $tweet;

$curl curl_init($twitterapi_url);
curl_setopt($curlCURLOPT_RETURNTRANSFER1);
$response curl_exec($curl);
curl_close($curl);

$json_content json_decode($responsetrue);
$json_content preg_replace"/\r|\n/"""$json_content );

$html $json_content['html'];
if(!empty($html)) {
$request $smcFunc['db_insert']('insert',
            
'tweet_cache',
            
array(
                
'tweetid' => 'int''html' => 'text'
            
),
            
array(
                
$tweetaddslashes($html)
           
),
            
array('tweetid','html')
        
);
echo '{"html" : "' addslashes($html) .'"}'
}
else {
echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (does not exist)</p>"}';
}


}

$_GET filter_input_array(INPUT_GETFILTER_SANITIZE_NUMBER_INT);
$qv  $_GET['id'];
if(!empty(
$qv)) {check_cache($qv,$smcFunc);} else {echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (blank ID)</p>"}';}

?>

3 - New Database table to store the data




And finally

4 - The call in display_template.php to load the script (jquery required obviously)

Code: [Select]
<script type="text/javascript" src="/*scrips directory*/embed_v2.js"></script> '
This then results in automatic tweet embedding with a post.





So obviously this is a start point. The code by Larri's admission is somewhat of a "hack" and can be refined by someone more familiar with SMF, and obviously it would then need to be packaged as a mod. Ideally someone who will be able to handle the upkeep of the modification as twitter adjusts its API etc.

Only requirement is that L'arri remains credited in the code as the original coder and any mod made from this code must be free, no paid mods from our code. 

Like i say, this is just the rough beginnings, but it is all there as working code, just needs refinement, packaging and development.
« Last Edit: October 31, 2014, 02:07:04 PM by dimspace »

Offline Rumpa

  • Jr. Member
  • **
  • Posts: 192
  • Gender: Female
  • I Love My Love
    • rumpa.das.549 on Facebook
    • Rumpa on GitHub
Re: Automatic Twitter Embedding - Code provided
« Reply #1 on: November 02, 2014, 08:09:58 AM »
I make it almost.

Offline Cylent1

  • Semi-Newbie
  • *
  • Posts: 46
    • Drummer Underground
Re: Automatic Twitter Embedding - Code provided
« Reply #2 on: November 02, 2014, 06:12:16 PM »
I would most def be interesting in a mod like this...

Offline dimspace

  • Jr. Member
  • **
  • Posts: 343
    • @dimspace on Twitter
    • Velorooms.com
Re: Automatic Twitter Embedding - Code provided
« Reply #3 on: November 11, 2014, 10:32:29 AM »
well the code is there and works, just needs someone to refine it an package it, possibly add an admin panel to enable/disable it.

oh.. i didnt actally say what it does properly..

you post the twitter link, and it automatically embeds the tweet for you

Offline DEK24

  • Charter Member
  • Jr. Member
  • *
  • Posts: 146
Re: Automatic Twitter Embedding - Code provided
« Reply #4 on: November 29, 2014, 12:45:09 AM »
This would be big time

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,850
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Automatic Twitter Embedding - Code provided
« Reply #5 on: November 29, 2014, 02:06:40 AM »
Package/modified and submitted.

Requires cURL extension in php installed. And includes jquery hosted on cloudfare cdn.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Cylent1

  • Semi-Newbie
  • *
  • Posts: 46
    • Drummer Underground
Re: Automatic Twitter Embedding - Code provided
« Reply #6 on: November 29, 2014, 02:57:19 PM »
Installed no problem!  Where are the options in admin panel if any?

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,850
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Automatic Twitter Embedding - Code provided
« Reply #7 on: November 29, 2014, 03:01:17 PM »
No options in the admin panel.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Cylent1

  • Semi-Newbie
  • *
  • Posts: 46
    • Drummer Underground
Re: Automatic Twitter Embedding - Code provided
« Reply #8 on: November 29, 2014, 05:07:18 PM »
How exactly do we use this? I'm lost! lol  :-[

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,850
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Automatic Twitter Embedding - Code provided
« Reply #9 on: November 29, 2014, 06:00:00 PM »
You just post the twitter tweet in a thread and it auto embeds
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Cylent1

  • Semi-Newbie
  • *
  • Posts: 46
    • Drummer Underground
Re: Automatic Twitter Embedding - Code provided
« Reply #10 on: November 29, 2014, 06:36:03 PM »
Are you suggesting to use the "embed Tweet" option on the bottom of the tweets?  If so, then it still isn't working for me.

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,850
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Automatic Twitter Embedding - Code provided
« Reply #11 on: November 29, 2014, 06:38:07 PM »
No post a tweet on your board example

https://twitter.com/SMFHacks/status/536211331255660544
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Antes

  • Evil Black Cat
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 8,976
  • Gender: Male
  • Black cat rulz!
    • XinYenFon on GitHub
    • merta on LinkedIn
    • @XinYenFon on Twitter
    • WoWSnips
Re: Automatic Twitter Embedding - Code provided
« Reply #12 on: December 29, 2014, 06:55:48 PM »
This mod still under review, you guys can try my mod: http://custom.simplemachines.org/mods/index.php?mod=3837

- It has no external JS library included
- Tweets got directly from Official API
- Support old fashion bbcode as well as auto-embedding.
Open for PAID works # Support Me

Current Project(s): [ WoWSnips ]
Past Project(s): [ ezPortal ] # [ Lunarfall ] # [ RDD (HTML5) ]

Offline mark87

  • Semi-Newbie
  • *
  • Posts: 10
  • Gender: Male
    • @@dud6887 on Twitter
    • Wisconsinsportstalk.com
Re: Automatic Twitter Embedding - Code provided
« Reply #13 on: January 03, 2015, 10:54:50 AM »
Why not just use twitterfeed. com  and skip the coding ? Couple good threads on it and it never gives me any issues ?

Offline Antes

  • Evil Black Cat
  • SMF Friend
  • SMF Hero
  • *
  • Posts: 8,976
  • Gender: Male
  • Black cat rulz!
    • XinYenFon on GitHub
    • merta on LinkedIn
    • @XinYenFon on Twitter
    • WoWSnips
Re: Automatic Twitter Embedding - Code provided
« Reply #14 on: January 03, 2015, 11:08:15 AM »
They are different things, that site takes from forum and sends to social media while those two mods embeds the tweets into the forums (on request not automated).
Open for PAID works # Support Me

Current Project(s): [ WoWSnips ]
Past Project(s): [ ezPortal ] # [ Lunarfall ] # [ RDD (HTML5) ]

Offline dimspace

  • Jr. Member
  • **
  • Posts: 343
    • @dimspace on Twitter
    • Velorooms.com
Re: Automatic Twitter Embedding - Code provided
« Reply #15 on: January 07, 2015, 12:52:30 PM »
Package/modified and submitted.

Requires cURL extension in php installed. And includes jquery hosted on cloudfare cdn.

thanks for doing this.

as said before, larri and i had done the coding and hard coded but neither of us know how to package smf mods.

Offline dimspace

  • Jr. Member
  • **
  • Posts: 343
    • @dimspace on Twitter
    • Velorooms.com
Re: Automatic Twitter Embedding - Code provided
« Reply #16 on: January 07, 2015, 12:54:58 PM »
Why not just use twitterfeed. com  and skip the coding ? Couple good threads on it and it never gives me any issues ?

the reasons for this code were two fold.

1) makes embedding tweets incredibly simple, you only need to include the tweet link

2) it was built to use the smf database for entries which means on very large forums with a lot of page views you dont get locked out of twitter (basically every time the tweet is viewed its a fresh view and twitter gets very upset if you make multiple views - which is the problem we had), this caches the tweets in the smf database.

Offline dimspace

  • Jr. Member
  • **
  • Posts: 343
    • @dimspace on Twitter
    • Velorooms.com
Re: Automatic Twitter Embedding - Code provided
« Reply #17 on: April 25, 2015, 04:40:20 PM »
- Tweets got directly from Official API

This is the problem, for very large forums you end up getting locked out by twitter for making too many calls to the api (we had loads of days where our embedded tweets just vanished when twitter locked us out), which is why an alternate solution was looked for.

Offline Xpresskonami

  • Jr. Member
  • **
  • Posts: 237
  • Gender: Male
Re: Automatic Twitter Embedding - Code provided
« Reply #18 on: May 12, 2017, 07:39:55 AM »
I install this Auto tweet Embeed. it works fine, but suddenly the mod stop working. please really need solutions to this

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,850
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Automatic Twitter Embedding - Code provided
« Reply #19 on: May 12, 2017, 08:29:33 AM »
Link to forum with a topic that has a twitter post.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro