Fawlty Опубликовано 3 октября, 2007 Жалоба Поделиться Опубликовано 3 октября, 2007 Есть граббер контента написаный на php, данный скрипт каждый час проверяет обновления на сайте жертвы и грабит его содержимое удаляя при этом старый заграбленный контент из базы данных MySQL. Вот код этого самого скрипта: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"><html><head><title>Grab and Save</title></head><body text="#000000" bgcolor="#FFFFFF" topmargin="0" leftmargin="0" rightmargin="0" bottommargin="0" marginheight="0" marginwidth="0"><? /* Grab and Save v. 1.1by Tom ChurmUpdated Oct. 7th, 2002 after Cisco changed their webpage;--)churmtom@hotmail.comhttp://www.churm.comGrab and Save is GPL software.The actual 'Grabber' part of this script is taken from"PHP Headlines Grabber" by Neil Moomey:http://px.sklar.com/code-pretty.html?code_id=442This script grabs content from another website and saves it ina MySQL database table, thus reducing page loading time in subsequently displaying this content.Currently it checks to see if the content is less than one hour old.If the content is more than one hour old, it will be leeched anew from thetarget website, displayed on your webpage, and saved in MySQL.If the content is less than one hour old, it'll just beselected from your MySQL table to be displayed on your web page (because this is faster!).Included is an option to delete older 'grabbed' content automatically fromyour database, thus saving database space.Also included is an option to check for new content only on a daily--rather than on an hourly--basis.This script is presented purely for educational purposes, as the leeching ofcontent from other websites may be illegal!Use this script at your own risk!===========================================How to Use this Script:1) create a table using the table structure dump included below2) edit all the variables starting in the section labeled 'Start of Configuration Variables'3) place this file on your server or copy the code into an existing page, etc..===========================================Interested in a desktop application that backs up aCOMPLETE MySQL database to an Excel File--without using ODBC?Then try out my MySQL Database 2 Excel KonvertR program:http://www.churm.com/konvertr/index.php*//* //Here's a MySQL Table-Structure Dump for the Table //where you can Save your Grabbed Content://(mine is saved in a database called 'grabbed')# Make a new database called 'grabbed'Create Database grabbed;## Make a new table called 'content1'#CREATE TABLE content1 ( ID int(10) unsigned NOT NULL auto_increment, date_grabbed int(255) NOT NULL default '0', content text, stamp timestamp(14) NOT NULL, PRIMARY KEY (ID), KEY ID (ID)) TYPE=MyISAM;*//* Start of Configuration Variables *///set this variable to 1 to check for new content hourly,//otherwise set it to 0 to check for new content just once a day$hourly = 1;//set this variable to 1 to delete older grabbed content from the database,//set it to 0 to leave the old content in the database.$delete_old_content = 1;//this variable is the url for the web page with the content that//you want to grab. if you don't know the file-ending of the page//(ie: .htm, .html, .php), be sure to include a Trailing Slash ("/")$url_to_grab = "http://newsroom.cisco.com/dlls/index.html";//these are the variables for the start and end of the//html content to grab//IMPORTANT: BOTH OF THESE MUST BE UNIQUE PIECES OF CODE FROM//THE TARGET WEBSITE--IF BOTH OF THESE ARE NOT UNIQUE,//THIS SCRIPT WILL NOT WORK!!!$start="<!-- end spotlight section --> <br>"; $end="<!-- End of the actual text content -->"; //MySQL Connection Definitions://change these to your connection details$DB_Server = "localhost"; //mysql database host$DB_Username = "root"; //mysql username$DB_Password = ""; //mysql password$DB_DBName = "grabbed"; //mysql database name$DB_TBLName = "content1"; //mysql table name/* End of Configuration Variables *///create a unix time stamp for today's date//uses mktime() function://mktime ( int hour, int minute, int second, int month, int day, int year );if($hourly == 1){//checks for new content every hour//this will ignore minutes & seconds$todaydate = mktime (date("H"),0,0,date("m"),date("d"),date("Y"));}else{//checks for new content once a day//this will ignore hours, minutes & seconds$todaydate = mktime (0,0,0,date("m"),date("d"),date("Y"));}/* Start - MySQL Connection Defs */Function MySQLConnect(){ $connection=@mysql_connect($DB_Server, $DB_Username, $DB_Password) or die(mysql_error());}Function MySQLQuery($DB_DBName,$query){ $success=@mysql_db_query($DB_DBName, $query) or die(mysql_error());}//alternative connection settings for selecting//mysql_fetch_array($result)$ALT_Connect = @mysql_connect($DB_Server, $DB_Username, $DB_Password) or die(mysql_error());$ALT_Db = @mysql_select_db($DB_DBName, $ALT_Connect)or die(mysql_error());/* End - MySQL Connection Defs *///go to database and get item for today's date if it exists$dep_list = "SELECT * FROM $DB_TBLName WHERE date_grabbed = $todaydate";$result1 = @mysql_query($dep_list, $ALT_Connect)or die("Couldn't execute query #1");//loop through departments table to populate select menu//set var to 'none' so we can check it later//we'll check if no content for today's date is found in the database$date_grabbed = "none";//start of db loopwhile ($row = mysql_fetch_array($result1)) {$date_grabbed = $row['$date_grabbed'];$grabbedContent = $row['content'];//strip slashes from content//this is important because addslashes() is used before//saving the content in the database$grabbedContent = stripslashes($grabbedContent);}//end of db loop//if there's no content for this date/time saved in the database, leech it,//then display it and save it in MySQLIf($date_grabbed=="none"){/* Start of Grabber */// PHP Headlines Grabber // Grab source code from a file or web site if(!($myFile=@fopen($url_to_grab,"r"))) { echo "The news interface is down for maintenance."; exit; } while(!feof($myFile)) { // Read each line and add to $myLine $myLine.=fgets($myFile,255); } fclose($myFile); $start_position=strpos($myLine, $start); $end_position=strpos($myLine, $end)+strlen($end); $length=$end_position-$start_position; $myLine=substr($myLine, $start_position, $length); //clean up some of the content: this will vary depending on the content//and what you need for your web page$myLine=ereg_replace("../images", "http://newsroom.cisco.com/images", $myLine);$myLine=ereg_replace("href=\"", "target=\"_blank\" href=\"http://newsroom.cisco.com/dlls/", $myLine);$myLine=ereg_replace("News Releases", "<b>Cisco News</b>", $myLine);$myLine=eregi_replace(" bgColor=\"#ffffff\"", "", $myLine);$myLine=eregi_replace(" bgColor=\"#336666\"", "", $myLine);// Display HTML echo $myLine;echo "<a href=\"http://newsroom.cisco.com/dlls/index.html\" target=\"_blank\">More Cisco News</a>...";/* End of Grabber *///add slashes to prevent problems with MySQL queries//don't forget to stripslashes() before displaying it!$myLine = addslashes($myLine);//insert the grabbed content into the databaseMySQLQuery($DB_DBName,"INSERT INTO $DB_TBLName (date_grabbed, content) VALUES ('$todaydate', '$myLine')");//if option is enabled to delete old content, go ahead and delete it from dbif( $delete_old_content == 1 ){MySQLQuery($DB_DBName,"DELETE FROM $DB_TBLName WHERE date_grabbed < $todaydate");}}else//this is for when there IS content for this date/time saved in the database//==> then simply display it{// Just Display HTML echo $grabbedContent;echo "<a href=\"http://newsroom.cisco.com/dlls/index.shtml\" target=\"_blank\">More Cisco News</a>...";}?> </body></html> Можно ли каким-нибудь образом модифицировать код скрипта что-бы он не автоматически проверял и грабил сайт а делал это вручную, допустим после ввода в адресную строку какого-нибудь параметра например "http://mysite.ru/index.php?update" Я работаю над региональным сайтом телеканала СТС, и граббер мне нужен для отображения на своём сайте фильмов которые выходят в эфир в 21:00 Ссылка на комментарий Поделиться на другие сайты Поделиться
Рекомендуемые сообщения
Для публикации сообщений создайте учётную запись или авторизуйтесь
Вы должны быть пользователем, чтобы оставить комментарий
Создать учетную запись
Зарегистрируйте новую учётную запись в нашем сообществе. Это очень просто!
Регистрация нового пользователяВойти
Уже есть аккаунт? Войти в систему.
Войти