Browse Source

first working version before php

Generates tiles, has ubuntu installer
master
poikilos 9 years ago
committed by Jacob Gustafson
parent
commit
6cd9fc9cef
  1. 3
      .gitignore
  2. 47
      README.md
  3. 3
      chunkymap-cronjob
  4. 305
      chunkymap-regen.py
  5. 78
      colors.txt
  6. 9
      install-chunkymap-on-ubuntu.sh
  7. 2
      mapper-pyarch.py
  8. 1070
      minetestmapper-numpy.py
  9. 3
      minetestserver-stop.sh
  10. 6
      set-minutely-crontab-job.sh

3
.gitignore

@ -1,3 +1,6 @@
# chunkymap generated data
chunkymapdata/
# Byte-compiled / optimized / DLL files # Byte-compiled / optimized / DLL files
__pycache__/ __pycache__/
*.py[cod] *.py[cod]

47
README.md

@ -1,2 +1,49 @@
# minetest-chunkymap # minetest-chunkymap
A Minetest online web live map generator not requiring mods, with emphasis on compatibility without regard to speed, efficiency, or ease of use. A Minetest online web live map generator not requiring mods, with emphasis on compatibility without regard to speed, efficiency, or ease of use.
Compatible with GNU/Linux systems, Windows, or possibly other systems (but on Windows, chunkymap-regen.py must be scheduled by hand with Scheduled Tasks)
License: GPLv3 (see LICENSE.txt and always include it and way to access your source code when copying your program)
This program comes without any warranty, to the extent permitted by applicable law.
## Requirements:
* A minetest version compatible with minetestmapper-numpy.py Made by Jogge, modified by celeron55
* Python 2.7 (any 2.7.x)
* Other requirements for Windows are below; other requirements for Ubuntu are installed by install-chunkymap-on-ubuntu.sh (for other distros, modify it and send me a copy as a GitHub issue as described below in the Installation section)
## Installation
(NOTE: map refresh skips existing tiles unless you delete the related png and text files in your chunkymapdata folder)
* change set-minutely-crontab-job.sh to replace "owner" with the user that has the minetest folder (with util folder under it, not .minetest)
* Install the git version of minetest (or otherwise install 0.4.13 or other version compatible with the map generators used by chunkymap)
* IF you are using Ubuntu go to a terminal, cd to this folder, then run
chmod +x install-chunkymap-on-ubuntu.sh && ./install-chunkymap-on-ubuntu.sh
otherwise first edit the file for your distro (and please send the modified file to me [submit as new issue named such as: DISTRONAME installer except instead of DISTRONAME put the distro you made work])
* IF you are using a distro such as Ubuntu 14.04 where first line of /etc/crontab is "m h dom mon dow user command" then if you want regular refresh of map then run
(otherwise first edit the script to fit your crontab then)
chmod +x set-minutely-crontab-job.sh && ./set-minutely-crontab-job.sh
* IF you are using Windows
* put these files anywhere
* manually schedule a task in Task Scheduler to run C:\Python27\python chunkymap-regen.py every minute
* python 2.7.x such as from python.org
* run mapper-pyarch.py to make sure you know whether to download the following in 32-bit or 64-bit
Administrator Command Prompt (to find it in Win 10, right-click windows menu)
* update python package system:
C:\python27\python -m pip install --upgrade pip wheel setuptools
* numpy such as can be installed via the easy unofficial installer wheel at
http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
then:
cd to the folder where you downloaded the whl file
C:\python27\python -m pip install "numpy-1.10.4+mkl-cp27-cp27m-win32.whl"
(but put your specific downloaded whl file instead)
* Pillow (instead of PIL (Python Imaging Library) which is a pain on Windows): there is a PIL installer wheel for Python such as 2.7 here:
http://www.lfd.uci.edu/~gohlke/pythonlibs/
as suggested on http://stackoverflow.com/questions/2088304/installing-pil-python-imaging-library-in-win7-64-bits-python-2-6-4
then:
C:\python27\python -m pip install "Pillow-3.1.1-cp27-none-win32.whl"
(but put your specific downloaded whl file instead, such as Pillow-3.1.1-cp27-none-win_amd64.whl)
* edit chunkymap_regen.py and uncomment website_root="/var/www/html/minetest" then change the value in quotes to your web server's htdocs folder such as, if you are using Apache, can be found as the value of the DocumentRoot variable in httpd.conf in the Apache folder in Program Files
* edit chunkymap_regen.py and change world_name to your world name
## Known Issues
* Make a php file that shows the map on an html5 canvas (refresh players every 10 seconds, check for new map chunks every minute)
* Make players invisible if they stay in one spot too long (consider them logged out by that method alone since not requiring mods)
* Detect failure of minetestmapper-numpy.py and instead use minetest-mapper if on linux, otherwise show error (since Windows has no minetest-mapper at least on client 0.4.13)

3
chunkymap-cronjob

@ -0,0 +1,3 @@
#!/bin/sh
# NOTE: only works since all scripts in /etc/cron.*/ or crontab run as root
python /home/owner/minetest/utils/chunkymap-regen.py

305
chunkymap-regen.py

@ -0,0 +1,305 @@
#!/usr/bin/env python2
import os
import subprocess
# REQUIRES: see README.md
#minetestmapper-numpy.py calculates the region as follows:
#(XMIN','XMAX','ZMIN','ZMAX'), default = (-2000,2000,-2000,2000)
#sector_xmin,sector_xmax,sector_zmin,sector_zmax = numpy.array(args.region)/16
#sector_ymin = args.minheight/16
#sector_ymax = args.maxheight/16
#region server-specific options
full_render = False # the preferred method of full render is deleting all files from the folder chunkymap_data_path such as /var/www/html/minetest/chunkymapdata (or chunkymap in current directory on Windows)
input_string = ""
username = "owner"
os_name="linux"
if (os.path.sep!="/"):
os_name="windows"
#input_string = input("Which username contains minetest/util/minetestmapper-numpy.py (minetest not .minetest) ["+username+"]?")
if (len(input_string)>0):
username = input_string
website_root="/var/www/html/minetest"
#input_string = input("What is the root folder of your minetest website ["+website_root+"]?")
if (len(input_string)>0):
website_root = input_string
world_name = "FCAGameAWorld"
#input_string = input("What is the game name ["+world_name+"]")
if (len(input_string)>0):
world_name = input_string
#region server-specific options
profiles_path = "/home"
if os_name=="windows":
profiles_path = "C:\\Users"
profile_path = os.path.join(profiles_path, username)
#if (not os.path.isdir(profile_path)):
# profile_path = os.path.join(profiles_path, "jgustafson")
dotminetest_path = os.path.join(profile_path,".minetest")
if (os_name=="windows"): dotminetest_path = "C:\\games\\Minetest"
worlds_path = os.path.join(dotminetest_path,"worlds")
world_path = os.path.join(worlds_path, world_name)
auto_chosen_world = False
if not os.path.isdir(world_path):
#for item in os.walk(worlds_path):
print "LOOKING FOR WORLDS IN " + worlds_path
for dirname, dirnames, filenames in os.walk(worlds_path):
index = 0
for j in range(0,len(dirnames)):
i = len(dirnames) - 0 - 1
if dirnames[i][0] == ".":
print " SKIPPING "+dirnames[i]
dirnames.remove_at(i)
for subdirname in dirnames:
print " EXAMINING "+subdirname
if (index == len(dirnames)-1): # skip first one because the one on my computer is big
world_name = subdirname
world_path = os.path.join(dirname, subdirname) # os.path.join(worlds_path, "try7amber")
print " CHOSE "+world_path
auto_chosen_world = True
break
index += 1
if auto_chosen_world:
break
python_exe_path = "python"
try:
alt_path = "C:\\python27\python.exe"
if os.path.isfile(alt_path):
python_exe_path = alt_path
except:
pass # do nothing, probably linux
mtmn_path = os.path.join( profile_path, "minetest/util/minetestmapper-numpy.py" )
colors_path = os.path.join( profile_path, "minetest/util/colors.txt" )
if os_name=="windows":
mtmn_path = os.path.join(os.path.dirname(__file__), "minetestmapper-numpy.py")
colors_path = os.path.join(os.path.dirname(__file__), "colors.txt")
website_root = os.path.dirname(__file__)
class MTChunk:
x = None
z = None
is_player_here = None
def __init__(self):
self.is_player_here = False
is_save_output_ok = True
def get_dict_from_conf_file(path,assignment_operator="="):
results = None
if os.path.isfile(path):
results = {}
ins = open(yaml_path, 'r')
line = True
operator = ":"
while line:
line = ins.readline()
if line and len(line)>0:
line_strip=line.strip()
if not line_strip[0]=="#": # if not comment
if not line_strip[0]=="-": # ignore yaml arrays
ao_index = line_strip.find(assignment_operator)
if ao_index>=1: # intentionally skip zero-length variable names
if ao_index<len(line_strip)-1: # skip yaml implicit nulls or yaml objects
result_name = line_strip[:ao_index]
result_value = line_strip[ao_index+1:]
print " CHECKING MAP..."+result_name+":"+result_value
results[result_name]=result_value
ins.close()
return results
if os.path.isfile(mtmn_path) and os.path.isfile(colors_path):
chunkymap_data_path=os.path.join(website_root,"chunkymapdata")
yaml_name = "metadata.yml"
yaml_path = os.path.join(chunkymap_data_path, yaml_name)
if not os.path.isdir(chunkymap_data_path):
os.mkdir(chunkymap_data_path)
chunkymap_players_name = "players"
chunkymap_players_path = os.path.join(chunkymap_data_path, chunkymap_players_name)
if not os.path.isdir(chunkymap_players_path):
os.mkdir(chunkymap_players_path)
players_path = os.path.join(world_path, "players")
for dirname, dirnames, filenames in os.walk(players_path):
for filename in filenames:
file_fullname = os.path.join(players_path,filename)
#print " EXAMINING "+filename
badstart_string = "."
player_name = None
player_position = None
if (filename[:len(badstart_string)]!=badstart_string):
ins = open(file_fullname)
line = True
has_enough_data = False
while line:
line = ins.readline()
if line:
ao_index = line.find("=")
if ao_index > 0:
ini_name = line[:ao_index].strip()
ini_value = line[ao_index+1:].strip()
if ini_name=="name":
player_name = ini_value
elif ini_name=="position":
player_position = ini_value
if (player_name is not None) and (player_position is not None):
has_enough_data = True
break
ins.close()
player_dest_path = os.path.join(chunkymap_players_path,filename+".yml")
if has_enough_data:
#if player_name!="singleplayer":
outs = open(player_dest_path, 'w')
outs.write("name:"+player_name+"\n") # python automatically uses correct newline for your os when you put "\n"
outs.write("position:"+player_position+"\n")
outs.close()
mapvars = get_dict_from_conf_file(yaml_path,":")
#is_testonly == (os_name=="windows")
if mapvars is not None and set(['world_name']).issubset(mapvars):
#print " (FOUND world_name)"
if mapvars["world_name"] != world_name:
print ("REMOVING data since from different world (map '"+str(mapvars["world_name"])+"' is not '"+str(world_name)+"')...")
for dirname, dirnames, filenames in os.walk(chunkymap_data_path):
index = 0
for j in range(0,len(filenames)):
i = len(filenames) - 0 - 1
if filenames[i][0] == ".":
print " SKIPPING "+filenames[i]
filenames.remove_at(i)
for filename in filenames:
file_fullname = os.path.join(chunkymap_data_path,filename)
print " EXAMINING "+filename
badstart_string = "chunk"
if (len(filename) >= len(badstart_string)) and (filename[:len(badstart_string)]==badstart_string):
os.remove(file_fullname)
elif filename==yaml_name:
os.remove(file_fullname)
chunks = {}
#region values to save to YAML
chunk_size = 80
chunkx_min = 0
chunkz_min = 0
chunkx_max = 0
chunkz_max = 0
total_generated_count = 0
#values for command arguments:
maxheight = 100
minheight = -50
pixelspernode = 1
#ALSO save to YAML:
#world_name
#world_path
#endregion values to save to YAML
square_generates_count = 1
while square_generates_count > 0:
square_generates_count = 0
for z in range (chunkz_min,chunkz_max+1):
for x in range(chunkx_min,chunkx_max+1):
#python ~/minetest/util/minetestmapper-numpy.py --region -1200 800 -1200 800 --drawscale --maxheight 100 --minheight -50 --pixelspernode 1 ~/.minetest/worlds/FCAGameAWorld ~/map.png
#sudo mv ~/map.png /var/www/html/minetest/images/map.png
#only generate the edges (since started with region 0 0 0 0) and expanding from there until no png is created:
is_outline = (x==chunkx_min) or (x==chunkx_max) or (z==chunkz_min) or (z==chunkz_max)
if is_outline:
chunk_luid = "x"+str(x)+"z"+str(z)
png_name = "chunk_"+chunk_luid+".png"
png_path = os.path.join(os.path.dirname(__file__), png_name)
x_min = x * chunk_size
x_max = x * chunk_size + chunk_size - 1
z_min = z * chunk_size
z_max = z * chunk_size + chunk_size - 1
cmd_suffix = ""
mapper_out_name = "chunk_"+chunk_luid+"_mapper_result.txt"
mapper_out_path = os.path.join(os.path.dirname(__file__), mapper_out_name)
if is_save_output_ok:
cmd_suffix = " > \""+mapper_out_path+"\""
#print "generating x = " + str(x_min) + " to " + str(x_max) + " , z = " + str(z_min) + " to " + str(z_max)
cmd_string = python_exe_path + " \""+mtmn_path + "\" --region " + str(x_min) + " " + str(x_max) + " " + str(z_min) + " " + str(z_max) + " --maxheight "+str(maxheight)+" --minheight "+str(minheight)+" --pixelspernode "+str(pixelspernode)+" \""+world_path+"\" \""+png_path+"\"" + cmd_suffix
dest_png_path = os.path.join(chunkymap_data_path, png_name)
dest_mapper_out_path = os.path.join(chunkymap_data_path, mapper_out_name)
is_empty_chunk = False
if os.path.isfile(dest_mapper_out_path):
ins = open(dest_mapper_out_path)
line = True
while line:
line = ins.readline()
if line:
line_strip = line.strip()
if "data does not exist" in line_strip:
is_empty_chunk = True
break
ins.close()
if full_render or ((not os.path.isfile(dest_png_path)) and (not is_empty_chunk)):
print cmd_string
subprocess.call(cmd_string, shell=True) # TODO: remember not to allow arbitrary command execution, which could happen if input contains ';' when using shell=True
if os.path.isfile(png_path):
total_generated_count += 1
square_generates_count += 1
try:
if (os.path.isfile(dest_png_path)):
os.remove(dest_png_path)
except:
print "Could not finish deleting '"+dest_png_path+"'"
try:
os.rename(png_path, dest_png_path)
except:
print "Could not finish moving '"+png_path+"' to '"+dest_png_path+"'"
try:
if (os.path.isfile(dest_mapper_out_path)):
os.remove(dest_mapper_out_path)
if is_save_output_ok:
os.rename(mapper_out_path, dest_mapper_out_path)
else:
if os.path.isfile(mapper_out_path):
os.remove(mapper_out_path)
except:
print "Could not finish deleting/moving output"
else:
if os.path.isfile(dest_png_path):
total_generated_count += 1
square_generates_count += 1
print("Skipping existing map tile " + png_name + " since not full_render")
elif is_empty_chunk:
print("Skipping empty chunk " + chunk_luid + " since not full_render")
print "" # blank line before next z so output is human readable
chunkx_min -= 1
chunkz_min -= 1
chunkx_max += 1
chunkz_max += 1
#end while square outline (1-chunk-thick outline) generated any png files
outs = open(yaml_path, 'w')
outs.write("world_name:"+str(world_name) + "\n")
outs.write("chunk_size:"+str(chunk_size) + "\n")
outs.write("pixelspernode:"+str(pixelspernode) + "\n")
outs.write("chunkx_min:"+str(chunkx_min) + "\n")
outs.write("chunkz_min:"+str(chunkz_min) + "\n")
outs.write("chunkx_max:"+str(chunkx_max) + "\n")
outs.write("chunkz_max:"+str(chunkz_max) + "\n")
#values for command arguments:
outs.write("maxheight:"+str(maxheight) + "\n")
outs.write("minheight:"+str(minheight) + "\n")
#ALSO save to YAML:
outs.write("world_path:"+str(world_path) + "\n")
outs.write("chunkymap_data_path:"+str(chunkymap_data_path) + "\n")
outs.write("total_generated_count:"+str(total_generated_count) + "\n")
outs.close()
else:
print "failed since this folder must contain colors.txt and minetestmapper-numpy.py"

78
colors.txt

@ -0,0 +1,78 @@
0 128 128 128 # CONTENT_STONE
2 39 66 106 # CONTENT_WATER
3 255 255 0 # CONTENT_TORCH
9 39 66 106 # CONTENT_WATERSOURCE
e 117 86 41 # CONTENT_SIGN_WALL
f 128 79 0 # CONTENT_CHEST
10 118 118 118 # CONTENT_FURNACE
15 103 78 42 # CONTENT_FENCE
1e 162 119 53 # CONTENT_RAIL
1f 154 110 40 # CONTENT_LADDER
20 255 100 0 # CONTENT_LAVA
21 255 100 0 # CONTENT_LAVASOURCE
800 107 134 51 # CONTENT_GRASS
801 86 58 31 # CONTENT_TREE
802 48 95 8 # CONTENT_LEAVES
803 102 129 38 # CONTENT_GRASS_FOOTSTEPS
804 178 178 0 # CONTENT_MESE
805 101 84 36 # CONTENT_MUD
808 104 78 42 # CONTENT_WOOD
809 210 194 156 # CONTENT_SAND
80a 123 123 123 # CONTENT_COBBLE
80b 199 199 199 # CONTENT_STEEL
80c 183 183 222 # CONTENT_GLASS
80d 219 202 178 # CONTENT_MOSSYCOBBLE
80e 70 70 70 # CONTENT_GRAVEL
80f 204 0 0 # CONTENT_SANDSTONE
810 0 215 0 # CONTENT_CACTUS
811 170 50 25 # CONTENT_BRICK
812 104 78 42 # CONTENT_CLAY
813 58 105 18 # CONTENT_PAPYRUS
814 196 160 0 # CONTENT_BOOKSHELF
815 205 190 121 # CONTENT_JUNGLETREE
816 62 101 25 # CONTENT_JUNGLEGRASS
817 255 153 255 # CONTENT_NC
818 102 50 255 # CONTENT_NC_RB
819 200 0 0 # CONTENT_APPLE
default:stone 128 128 128
default:stone_with_coal 50 50 50
default:water_flowing 39 66 106
default:torch 255 255 0
default:water_source 39 66 106
default:sign_wall 117 86 41
default:chest 128 79 0
default:furnace 118 118 118
default:fence_wood 103 78 42
default:rail 162 119 53
default:ladder 154 110 40
default:lava_flowing 255 100 0
default:lava_source 255 100 0
default:dirt_with_grass 107 134 51
default:tree 86 58 31
default:leaves 48 95 8
default:dirt_with_grass_and_footsteps 102 129 38
default:mese 178 178 0
default:dirt 101 84 36
default:wood 104 78 42
default:sand 210 194 156
default:cobble 123 123 123
default:steelblock 199 199 199
default:glass 183 183 222
default:mossycobble 219 202 178
default:gravel 70 70 70
default:sandstone 204 0 0
default:cactus 0 215 0
default:brick 170 50 25
default:clay 104 78 42
default:papyrus 58 105 18
default:bookshelf 196 160 0
default:jungletree 205 190 121
default:junglegrass 62 101 25
default:nyancat 255 153 255
default:nyancat_rainbow 102 50 255
default:apple 200 0 0
default:desert_sand 210 180 50
default:desert_stone 150 100 30
default:dry_shrub 100 80 40

9
install-chunkymap-on-ubuntu.sh

@ -0,0 +1,9 @@
#!/bin/sh
sudo apt-get install python-numpy python-pil
cd ~
rm -f ~/minetestmapper-numpy.py
wget https://github.com/spillz/minetest/raw/master/util/minetestmapper-numpy.py
#since colors.txt is in ~/minetest/util:
mv minetestmapper-numpy.py ~/minetest/util/minetestmapper-numpy.py
cp
# NOTE: colors.txt should ALREADY be in ~/minetest/util

2
mapper-pyarch.py

@ -0,0 +1,2 @@
import platform
print platform.architecture()[0]

1070
minetestmapper-numpy.py

File diff suppressed because it is too large

3
minetestserver-stop.sh

@ -0,0 +1,3 @@
#!/bin/sh
#TODO: set pid then:
# ssh hostname 'kill -TERM $pid'

6
set-minutely-crontab-job.sh

@ -0,0 +1,6 @@
#!/bin/sh
sudo su -
# NOTE: this works only since user is a field on Ubuntu (on some GNU/Linux systems it is not, which is implied by omission at http://www.adminschoice.com/crontab-quick-reference)
# Minute, Hour, Day of Month, Month (1 to 12), Day of Week
# m h dom mon dow user command
echo "* * * * * root /home/owner/minetest/utils/chunkymap-cronjob" >> /etc/crontab
Loading…
Cancel
Save