python3字符串自增

週末喝恆大冰泉,發現掃二維碼中獎概率頗高,而且可以中獎金額直接用來給手機充值或提現到銀行卡,做的比較好。趁理髮無聊的時候寫了段python掃了一會兒,但是空手而歸。代碼也很簡單和用Python抓取大衆點評的用戶評論差不多,主要就是字符串自增這塊用了個遞歸:

from bs4 import BeautifulSoup import string,sys,time,random,urllib,http.cookiejar,socket #url="https://sao.so/t/dSTUVvjUhPHpz" baseDir="/storage/sdcard0/com.hipipal.qpyplus/scripts3/test/" base="_0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" HDcode="dSTUVvjUhPIjL" bingoNo=0 failNo=0 def increase(s): sHead=s[0:-1] sFoot=s[-1:] sNew="" if(sFoot != "Z"): index=base.index(sFoot) sFootNew=base[index+1] sNew=sHead+sFootNew else: sHeadNew=increase(sHead) sFootNew="_" sNew=sHeadNew+sFootNew return sNew def getHTML(url): request = urllib.request.Request(url) request.add_header("User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0") try: response = urllib.request.urlopen(request) except (urllib.error.HTTPError, socket.error,urllib.error.URLError) as e: print('Connection error occurred when inserting data.'+str(e)) else: if response.code != 200: print("Error code:"+response.code) else: html = response.read().decode('utf-8') return html def check(html): global bingoNo global failNo #with open(baseDir+'expired.html',encoding='utf-8') as html: soup = BeautifulSoup(html) images = soup.find_all('img') for image in images: if "errorQrCode.jpg" in image.get('src') : failNo = failNo + 1 print("fake: " + HDcode + " " + str(failNo)) break elif "冰泉君已经失身啦" in soup.get_text(): print("expired" + HDcode) break else: print("bingo" + HDcode) bingoNo = bingoNo + 4 if __name__ == '__main__': #html=getHTML(url) #with open(baseDir+'fake.html', mode='w', encoding='utf-8') as html_file: #html_file.write(html) global HDcode while(bingoNo < 2): url = "https://sao.so/t/" + HDcode html = getHTML(url) check(html) HDcode = increase(HDcode) time.sleep(random.randrange(4,7)) [/code]

其實命中的概率真的非常低,拿13位編碼來說,一共有(11+26+26)^13=2.46*10^23種組合,瓶子上說中獎概率33.3%,按300億銷售目標和5元均價算,一共有2×10^10瓶是可以中獎的。除一下就發現……不用除了吧

用Python抓取大衆點評的用戶評論

大衆點評的知识产权声明可真是霸道啊!還是自己先保存一份。下面代碼先將評論及商戶保存到sqlite數據庫,如果需要還可以導出成CSV,這樣辦公軟件就能直接打開查看了。

from bs4 import BeautifulSoup
import sys,time,random,urllib,http.cookiejar,socket,sqlite3,csv


goOn=1
stopDate=''
UserID=''
review={'shopName':'','shopAddr':'','shopURL':'','reviewURL':'','star':'',
'starDetail':'','costPerPeople':'','reviewText':'','dishes':'','reviewTime':''}

def getHTML(url):
  print("Fetching "+url)
  request = urllib.request.Request(url)
  request.add_header("User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0")
  try:
    response = urllib.request.urlopen(request)
  except (urllib.error.HTTPError, socket.error,urllib.error.URLError) as e:
    print('Connection error occurred when inserting data.'+str(e))
  else:
    if response.code != 200:
      print("Error code:"+response.code)  
    else:
      html = response.read().decode('utf-8')
      return html

def getList(url):
  global review,goOn
  reviewList=getHTML(url)
  soupAll = BeautifulSoup(reviewList).find_all("div",{"class":"txt J_rptlist"})

  for soup in soupAll:
    shopLink = soup.find("a",{"class":"J_rpttitle"})
    review['shopName']=shopLink.text
    review['shopURL']=shopLink.get("href")
    
    shopAddr = soup.find("p",{"class":"col-exp"})
    review['shopAddr']=shopAddr.text
    
    reviewID = soup.find("a",{"class":"J_flower aheart"})
    review['reviewURL']="http://www.dianping.com/review/"+reviewID.get("data-id")
    
    reviewDateDiv = soup.find("div",{"class":"mode-tc info"})
    reviewDateSpan=reviewDateDiv.find("span",{"class":"col-exp"})
    reviewDate=str(reviewDateSpan.text)[3:]
    if(len(reviewDate)==8 and reviewDate>stopDate):
      getReview(review['reviewURL'])
      #抓取頻率
      time.sleep(random.randrange(5,10))
    else:
      goOn=0
  if(goOn==0):
    print("Finished.")
    exit()
    
      
    

def save():
  global review,UserID
  conn = sqlite3.connect('DZDB_'+UserID+'_Reviews.db')
  c = conn.cursor()
  c.execute("""create table if not exists reviews (ID integer primary key not NULL,shopName char(50),shopAddr char(100),shopURL char(100),reviewURL char(100),star char(1),starDetail char(15),costPerPeople char(15),reviewText TEXT,dishes char(100),reviewTime char(20))""")
  s="""insert into reviews (ID,shopName,shopAddr,shopURL,reviewURL,star,starDetail,costPerPeople,reviewText,dishes,reviewTime) VALUES (NULL,\'"""+review['shopName']+'\',\''+review['shopAddr']+'\',\''+review['shopURL']+'\',\''+review['reviewURL']+'\',\''+str(review['star'])+'\',\''+review['starDetail']+'\',\''+review['costPerPeople']+'\',\''+review['reviewText']+'\',\''+review['dishes']+'\',\''+review['reviewTime']+'\')'
  c.execute(s)
  conn.commit()
  c.close
  print("Record at "+review['shopName']+" saved to Datebase.")
  review={'shopName':'','shopAddr':'','shopURL':'','reviewURL':'','star':'',
'starDetail':'','costPerPeople':'','reviewText':'','dishes':'','reviewTime':''}

def getReview(url):
  global review
  reviewHTML=getHTML(url)
  reviewAll=BeautifulSoup(reviewHTML)
  shopInfo= reviewAll.find("ul",{"class":"contList-info"})
  star=str(shopInfo.find("li"))
  if("msstar50" in star):
    review['star']=5
  elif ("msstar40" in star):
    review['star']=4
  elif ("msstar30" in star):
    review['star']=3
  elif ("msstar20" in star):
    review['star']=2
  elif ("msstar10" in star):
    review['star']=1
  else:
    review['star']=0
  starDetails=shopInfo.find_all("span",{"class":"rst"})
  starDetail=""
  for s in starDetails:
    s1=s.text[0:3]
    starDetail=starDetail+s1
  review['starDetail']=starDetail
  
  reviewText= reviewAll.find("div",{"class":"contList-con"})
  review['reviewText']=reviewText.text
  units= reviewAll.find_all("div",{"class":"comment-unit"})
  for unit in units:
    unit=str(unit.text).replace('\n','')
    if("人均:" in unit):    
      review['costPerPeople']=unit[4:]
    elif("喜欢的菜:" in unit): 
      unit=unit.replace(' ','')
      unit=unit.replace('\xa0',' ')
      review['dishes']=unit[7:]
    
  reviewInfo= reviewAll.find("ul",{"class":"contList-fn"})  
  reviewTime=reviewInfo.find("li")
  review['reviewTime']=reviewTime.text
  save() 

def main():
  fun=int(input("请输入数字选择功能:\n[1]抓取数据,[2]导出数据: \n"))
  if(fun==1):
    fetchReview()
  elif(fun==2):
    sqliteToCSV()
  else:
    print("请输入1或2。")

    
def sqliteToCSV():
  dbFile=str(input("请输入数据库文件名:\n"))
  with open(dbFile+'.csv','w+',newline='') as csvfile:
    spamwriter = csv.writer(csvfile)
    conn=sqlite3.connect(dbFile)
    c = conn.cursor()
    spamwriter.writerow(['ID','shopName','shopAddr','shopURL','reviewURL','star',
'starDetail','costPerPeople','reviewText','dishes','reviewTime'])
    for row in c.execute('SELECT * FROM reviews'):
      spamwriter.writerow(row)
    c.close()
    print("CSV文件成功導出。")
    
def fetchReview():
  #抓取参数:用户ID,起始页,结束日期
  global stopDate,UserID
  UserID=str(input("请输入您的大众点评ID,可以在您大众点评主页的网址中看到,如23262500:\n"))
  startPageNo=int(input("开始的页码,如1:\n"))
  stopDate=str(input("请输入评论结束日期(yy-mm-dd),如00-00-00:\n"))
  
  urlBase="http://www.dianping.com/member/"+UserID+"/reviews?pg="
  startPageNo=startPageNo-1
  while(goOn==1):
    startPageNo=startPageNo+1
    getList(urlBase+str(startPageNo))
    
if __name__ == "__main__":
    main()
幾點說明
  • 抓取頻率不要過大,否則大衆點評會屏蔽IP。我在抓取到20頁左右的時候碰到過一次屏蔽IP。如果意外中斷,你可以設置參數繼續下載,附w3school的SQL基礎教程
  • BeautifulSoup真是個好工具,連Qpython3都自帶了,但是遺憾的是這個代碼在Qpython3上跑報NoneType錯誤。
  • 我用了幾次都沒問題。

IP地址和歸屬地

移動2G網速好爛!看個IP歸屬地都要等好久。於是自己寫了個小小的php,輸出用不了1K數據,這樣既省流量又快速。代碼在Github:根據IP查看地理位置


//服務端獲取客戶端IP
$ip = getenv('HTTP_CLIENT_IP')?:
    getenv('HTTP_X_FORWARDED_FOR')?:
    getenv('HTTP_X_FORWARDED')?:
    getenv('HTTP_FORWARDED_FOR')?:
    getenv('HTTP_FORWARDED')?:
    getenv('REMOTE_ADDR');

quick cocos2d win7下环境搭建

所需软件:

  • Win7 x64
  • Quick-Cocos2d-x v3.2-RC1,正常安装即可。
  • JDK,我用的是1.7.0_65,这个最新版应该也可以。安装后设置环境变量,可参考Ubuntu安装SunJava
  • adt-bundle-windows-x86_64-20140702,打开eclipse,帮助-安装新文件填入http://download.eclipse.org/koneki/releases/stable,我是把搜到的都安装了。另外首次创建虚拟机的时候可能要连外网,我用的Psiphon,顺利下载了4.4.4的镜像。有一个很好用的代理mirrors.neusoft.edu.cn:80,速度超快,推荐用这个!记得勾上强制使用http的选项然后重启Android SDK Manager。
  • android-ndk-r9d-windows-x86_64,必须是r9d,因为还不支持r10。SDK 和 NDK 不能放在包含中文和空格的目录中。 SDK/NDK 必须和 quick 的文件放在同一个分区中。请参考编译 Android 工程

安装完成后,打开桌面上的player3,新建项目.选择位置输入包名,选择方向即可。将X:\\cocos\quick-cocos2d-x-3.2rc0\cocos\platform\android\java\src下的org文件夹复制到你项目的src下。也可先将这个导入工作区,然后在自己项目中引用,详细参见Quick-Coco2d-x开发环境搭建。然后运行proj.android文件夹中的build_native.bat。最后打开adt中的eclipse,导入安卓项目,选中proj.android导入。点菜单栏中的运行-运行就可以在虚拟机中调试了。

关于导出apk可参考在eclipse中将android项目生成apk并且给apk签名。需要一提的是,要在项目文件上右键,导出。

Js AES加解密

密码:
原文:

密文:

加密js用的是Jeff.Mott.OR的crypto-js

在线小六壬

公曆:

如果輸入框不支持時間選擇,請以2015-05-19T10:36格式輸入查詢。

Google Geocoding API好用

之前用qpython写了个小程序记录地理位置,反向地理编码(就是根据经纬度查询实际位置)用的是SL4A的Android模块,它就是调用底层谷歌的方法查询。但是现在一谷歌不稳定,二新手机没有装谷歌框架,所以完全无法反向地理编码。今天看了下谷歌的Geocoding API,用php转发了一下获取到的json,用起来就方便多了。而且谷歌的这个服务不用繁琐的验证,不限制客户端,甚至连帐号都不用,真是良心啊!当然由于“地理编码是一项既耗时又耗资源的任务”,所以每天发送的地理位置查询请求不得超过 2,500 个,这对于少量用户足够了。代码可以到右上角github里下载。