Quantcast
Channel: RaGEZONE - MMO Development Forums
Viewing all articles
Browse latest Browse all 31884

gtop100 scraper (updated) [java 8!!]

$
0
0
uses JSoup, make sure to get it there http://jsoup.org/

if you don't know what this is, you don't need it and it isn't for you

PHP Code:

        public static final String URL "http://www.gtop100.com/maplestory/";
    public static final List<
GtopEntryentries = new ArrayList<>();

    public static 
void main(String[] args) {
        try {
            
Document doc Jsoup.connect(URL).get();
            
doc.getElementsByClass("list-reviews").select("li").forEach(-> {
                
Element divscore c.select("div[class$=div-score]").first();
                
Elements divextrawarp c.select("div[class$=extra-wrap]");
                
String title divextrawarp.first().select("span[class$=col14]").text();
                
String description divextrawarp.first().select("p").text();
                
int in Integer.parseInt(divscore.childNodes().get(0).outerHtml().replaceAll("[^\\d.]"""));
                
int out Integer.parseInt(divscore.childNodes().get(2).outerHtml().replaceAll("[^\\d.]"""));
                
int rank Integer.parseInt(divscore.childNodes().get(1).outerHtml().replaceAll("[^\\d.]"""));

                
entries.add(new GtopEntry(titledescriptionrankinout));
            });
        } catch (
IOException e) {
            
e.printStackTrace();
        }

        
entries.forEach(-> {
            
System.out.println(c.toString());
        });
    }

    static class 
GtopEntry {
        public final 
String titledescription;
        public final 
int rankinout;

        public 
GtopEntry(String titleString descriptionint rankint inint out) {
            
this.title title;
            
this.description description;
            
this.rank rank;
            
this.in in;
            
this.out out;
        }

        [
MENTION=2000004426]Override[/MENTION]
        public 
String toString() {
            return 
"GtopEntry{" +
                    
"title='" title '\'' +
                    
", description='" description '\'' +
                    
", rank=" rank +
                    
", in=" in +
                    
", out=" out +
                    
'}';
        }
    } 

adapt to your needs
enjoy

if you used my old scraper I released before, here is some compatible code
Spoiler:

replace your scrap method with this

PHP Code:

        public static Map<IntegerGtopEntryscrap() {
        try {
            
Map<IntegerGtopPositiongtop100 = new HashMap<>();
            
Document doc doc Jsoup.connect(URL).get();

            
int[] = { }; //i felt physical pain when i wrote this
            
doc.getElementsByClass("list-reviews").select("li").forEach(-> {
                
Element divscore c.select("div[class$=div-score]").first();
                
Elements divextrawarp c.select("div[class$=extra-wrap]");
                
String title divextrawarp.first().select("span[class$=col14]").text();
                
int in Integer.parseInt(divscore.childNodes().get(0).outerHtml().replaceAll("[^\\d.]"""));
                
int out Integer.parseInt(divscore.childNodes().get(2).outerHtml().replaceAll("[^\\d.]"""));
                
int rank Integer.parseInt(divscore.childNodes().get(1).outerHtml().replaceAll("[^\\d.]"""));

                
gtop100.put(++i[0], new GtopPosition(ranktitleinout));
            });

                        return 
gtop100;
        } catch (
IOException e) {
            
e.printStackTrace();
        }
                
                return 
null;
    } 

untested btw, idc really about my dumb old shit

Viewing all articles
Browse latest Browse all 31884

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>